subject:"Re\: \[R\-pkg\-devel\] Handling Not\-Always\-Needed Dependencies\?"

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies? - Part 2

2016-08-04 Thread Paul Gilbert




On 08/04/2016 11:51 AM, Dirk Eddelbuettel wrote:


On 4 August 2016 at 11:46, Paul Gilbert wrote:
| If my package has a test that needs another package, but that package is
| not needed in the /R code of my package, then I indicate it as
| "Suggests", not as "Depends" nor as "Imports".  If that package is not
| available when I run R CMD check, should the test pass?

Wrong question.

Better question:  Should the test be running?  My preference is for only
inside of a requireNamespace() (or equivalent) block as the package is not
guaranteed to be present.  In theory.


At the level of R CMD check throwing an error or not, I think this is 
arguing that it should be possible to pass the tests (not throw an 
error) even though they are not run, isn't it? (So your answer to my 
question is yes, at least the way I was thinking of the question.) Or do 
you mean you would just like the tests to fail with a more appropriate 
error message? Or do you mean, as Duncan suggests, that the person 
writing the test should be allowed to code in something to decide if the 
test is really important or not?




In practice people seem to unconditionally install it anyway, and think that
is a good idea.  I disagree on both counts but remain in the vocal minority.


Actually, I think you are in agreement with Uwe and Duncan on this 
point, Duncan having added the refinement that the test writer gets to 
decide. No one so far seems to be advocating for my position that the 
tests should necessarily fail if they cannot be run. So I guess I am the 
one in the minority.


Paul


Dirk



__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies? - Part 2

2016-08-04 Thread Duncan Murdoch


On 04/08/2016 12:55 PM, Bill Denney wrote:

On 8/4/2016 11:51 AM, Dirk Eddelbuettel wrote:

On 4 August 2016 at 11:46, Paul Gilbert wrote:
| If my package has a test that needs another package, but that package is
| not needed in the /R code of my package, then I indicate it as
| "Suggests", not as "Depends" nor as "Imports".  If that package is not
| available when I run R CMD check, should the test pass?

Wrong question.

Better question:  Should the test be running?  My preference is for only
inside of a requireNamespace() (or equivalent) block as the package is not
guaranteed to be present.  In theory.

In practice people seem to unconditionally install it anyway, and think that
is a good idea.  I disagree on both counts but remain in the vocal minority.

As another package maintainer, I had almost the identical question
reading the previous (long) thread, but the three answers here don't
give the same answer.  My question I can make even more concrete:

I use the testthat package for my testing.  I never use it in the R code
itself, and it is explicitly only used for testing.  Should that be
included as "Depends" because every test requires it or "Suggests"
because no end user ever needs it?

If "Depends", then it leads to over-installation of the package by end
users who don't care about running tests locally.  If "Suggests", then
all of the tests would fail (assuming that Dirk's suggestion is
implemented).


I'd say you should use Suggests, and test for its presence at the start 
of your test scripts, e.g.


if (!require("testthat"))
stop("These tests need testthat")

or

if (!requireNamespace("testthat"))
stop("these tests need testthat")

(The latter means you'd need to prefix all testthat functions with 
"testthat::", but it has the advantage that their names don't conflict 
with yours.)


Or perhaps you don't want to give an error, you just want to skip some 
of your tests.  It's your decision.


Duncan Murdoch

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies? - Part 2

2016-08-04 Thread Uwe Ligges




On 04.08.2016 18:55, Bill Denney wrote:

On 8/4/2016 11:51 AM, Dirk Eddelbuettel wrote:

On 4 August 2016 at 11:46, Paul Gilbert wrote:
| If my package has a test that needs another package, but that
package is
| not needed in the /R code of my package, then I indicate it as
| "Suggests", not as "Depends" nor as "Imports".  If that package is not
| available when I run R CMD check, should the test pass?

Wrong question.

Better question:  Should the test be running?  My preference is for only
inside of a requireNamespace() (or equivalent) block as the package is
not
guaranteed to be present.  In theory.

In practice people seem to unconditionally install it anyway, and
think that
is a good idea.  I disagree on both counts but remain in the vocal
minority.

As another package maintainer, I had almost the identical question
reading the previous (long) thread, but the three answers here don't
give the same answer.  My question I can make even more concrete:

I use the testthat package for my testing.  I never use it in the R code
itself, and it is explicitly only used for testing.  Should that be
included as "Depends" because every test requires it or "Suggests"
because no end user ever needs it?

If "Depends", then it leads to over-installation of the package by end
users who don't care about running tests locally.  If "Suggests", then
all of the tests would fail (assuming that Dirk's suggestion is
implemented).


Suggests.

Best,
Uwe Ligges




At a loss,

Bill

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel


__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies? - Part 2

2016-08-04 Thread Bill Denney


On 8/4/2016 11:51 AM, Dirk Eddelbuettel wrote:

On 4 August 2016 at 11:46, Paul Gilbert wrote:
| If my package has a test that needs another package, but that package is
| not needed in the /R code of my package, then I indicate it as
| "Suggests", not as "Depends" nor as "Imports".  If that package is not
| available when I run R CMD check, should the test pass?

Wrong question.

Better question:  Should the test be running?  My preference is for only
inside of a requireNamespace() (or equivalent) block as the package is not
guaranteed to be present.  In theory.

In practice people seem to unconditionally install it anyway, and think that
is a good idea.  I disagree on both counts but remain in the vocal minority.
As another package maintainer, I had almost the identical question 
reading the previous (long) thread, but the three answers here don't 
give the same answer.  My question I can make even more concrete:


I use the testthat package for my testing.  I never use it in the R code 
itself, and it is explicitly only used for testing.  Should that be 
included as "Depends" because every test requires it or "Suggests" 
because no end user ever needs it?


If "Depends", then it leads to over-installation of the package by end 
users who don't care about running tests locally.  If "Suggests", then 
all of the tests would fail (assuming that Dirk's suggestion is 
implemented).


At a loss,

Bill

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies? - Part 2

2016-08-04 Thread Duncan Murdoch


On 04/08/2016 11:46 AM, Paul Gilbert wrote:

(One question from the thread Handling Not-Always-Needed Dependencies?)

I hope not to start another long tangled thread, but I have a basic
confusion which I think has a yes/no answer and I would like to know if
there is agreement on this point (or is it only me that is confused as
usual).

If my package has a test that needs another package, but that package is
not needed in the /R code of my package, then I indicate it as
"Suggests", not as "Depends" nor as "Imports".  If that package is not
available when I run R CMD check, should the test pass?

Yes or no:  ?

(I realize my own answer might be different if the package was used in
an example or demo in place of a test, but that is just the confusion
caused by too many uses for Suggests. In the case of a test, my own
thought is that the test must fail, so my own answer is no. If the test
does not fail then there is no real testing being done, thus missing
code coverage in the testing. If the answer is no, then the tests do not
need to be run if the package is not available, because it is known that
they must fail. I think that not bothering to run the tests because the
result is known is even more efficient than other suggestions. I also
think it is the status quo.)

Hoping my confusion is cleared up, and this does not become another long
tangled thread,


I'd say it's up to you as the author of the test.  Would skipping that 
test mean that your package was not adequately tested?  If so, then you 
should get an error if it isn't available, because otherwise people will 
think they've done adequate testing when they haven't.  One way this 
could happen if a major function of your package is being tested on a 
sample  dataset from a Suggested package.  Users of your package don't 
need the other one, but testers do.


Duncan Murdoch

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies? - Part 2

2016-08-04 Thread Uwe Ligges




On 04.08.2016 17:46, Paul Gilbert wrote:

(One question from the thread Handling Not-Always-Needed Dependencies?)

I hope not to start another long tangled thread, but I have a basic
confusion which I think has a yes/no answer and I would like to know if
there is agreement on this point (or is it only me that is confused as
usual).

If my package has a test that needs another package, but that package is
not needed in the /R code of my package, then I indicate it as
"Suggests", not as "Depends" nor as "Imports".  If that package is not
available when I run R CMD check, should the test pass?

Yes or no:  ?


Yes, as the package should pass the checks if suggested packages are 
unavailable.
BUT if these are available and the code is wrong, then it should 
generate an error.


Best,
Uwe





(I realize my own answer might be different if the package was used in
an example or demo in place of a test, but that is just the confusion
caused by too many uses for Suggests. In the case of a test, my own
thought is that the test must fail, so my own answer is no. If the test
does not fail then there is no real testing being done, thus missing
code coverage in the testing. If the answer is no, then the tests do not
need to be run if the package is not available, because it is known that
they must fail. I think that not bothering to run the tests because the
result is known is even more efficient than other suggestions. I also
think it is the status quo.)

Hoping my confusion is cleared up, and this does not become another long
tangled thread,
Paul

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel


__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies? - Part 2

2016-08-04 Thread Dirk Eddelbuettel


On 4 August 2016 at 11:46, Paul Gilbert wrote:
| If my package has a test that needs another package, but that package is 
| not needed in the /R code of my package, then I indicate it as 
| "Suggests", not as "Depends" nor as "Imports".  If that package is not 
| available when I run R CMD check, should the test pass?

Wrong question.

Better question:  Should the test be running?  My preference is for only
inside of a requireNamespace() (or equivalent) block as the package is not
guaranteed to be present.  In theory.

In practice people seem to unconditionally install it anyway, and think that
is a good idea.  I disagree on both counts but remain in the vocal minority.

Dirk

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

2016-08-04 Thread Duncan Murdoch


On 03/08/2016 9:15 PM, Dirk Eddelbuettel wrote:


On 3 August 2016 at 17:35, Duncan Murdoch wrote:
| No, you are reading something into my messages that isn't there.  I
| think I understand your suggestion now (CRAN should test with no
| Suggested packages present, not with all of them present).  That is less
| work than they currently do.

And have Suggests:'ed package tested conditionally.  Whether their tests
execute will be in the control of the one running the tests because he can
make use the optionality provided by Suggests: Either install, or don't.

So it can be the same tests as now.  But better. Because "choice".

| I disagree with your suggestion for two reasons.
|
| 1. I think it would lead to less effective testing.

Nope. See above.

| 2. It would be disruptive during the transition.

How to get from here to there is left as an exercise to the reader.

The mostly-wasteful discussion has left me less than motivated to work on
this as there is just so much misunderstanding and blockage.  We could work
all this out during a hackathon day.  Over email, alas, things don't seem to
work too well.



Okay, I'll drop out now too.

Duncan Murdoch

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

2016-08-03 Thread Duncan Murdoch


On 03/08/2016 2:03 PM, Dirk Eddelbuettel wrote:


On 3 August 2016 at 13:44, Duncan Murdoch wrote:
| On 03/08/2016 11:17 AM, Dirk Eddelbuettel wrote:
| >
| > On 3 August 2016 at 16:21, Uwe Ligges wrote:
| > |
| > |
| > | On 03.08.2016 16:14, Dirk Eddelbuettel wrote:
| > | >
| > | > On 3 August 2016 at 16:00, Uwe Ligges wrote:
| > | > | On 03.08.2016 14:24, Dirk Eddelbuettel wrote:
| > | > | > Then again, users of TravisCI know that just toggling
| > | > | >
| > | > | > _R_CHECK_FORCE_SUGGESTS_=FALSE
| > | > |
| > | > | I was travelling, hence a delayed response:
| > | > |
| > | > | Why users of TravisCI? It is documented in the manual. Setting it to
| > | >
| > | > Because Travis breaks your check as it works in a cleanroom with only 
the
| > | > specified packages installed. As it (and that is the gist of my 
argument)
| > | > should ...
| > |
| > | No, because if I suggest a package, I want to check also the code that
| > | uses a suggested package. So I have to have the package installed.
| >
| > But then you are treating Suggests as Depends and installing irregardless.
|
| No, it is different from Depends.
|
| Depends implies something will be attached to the search list.  Suggests
| doesn't.  It is more like Imports, but without importing anything.

I am aware of the distinction.  I was talking, or trying to talk, about
'conditional' versus 'unconditional' "presence of packages" if you wish.

Depends/Import are the latter, Suggests is the former.

Or it would be if we chose to treat it like that. My gripe is that we don't.

| CRAN chooses to test in an environment where all Suggested packages are
| available.  They could do more testing (and I am pretty strongly in
| favour of that), e.g. *also* test in an environment where none of them
| are available.  But lots of packages will fail those tests, hence my
| request to you (not a gambit, I'm not playing a game) to estimate how
| many, and work out how to fix those packages.

I did send a PR to one such package in the past. The package maintainers
ignored it.  So I stopped.

I think I'll do the same with this thread now.

| Doing more testing will make it harder to get a package accepted on
| CRAN, but hopefully will mean that if it makes it there, it will be of
| better quality.
|
| The other thing I am trying to do is to work out ways to spread out the
| work that CRAN does to other people.  I believe the current model will
| not last another year, and I'd rather that it didn't collapse
| completely.  This is unlikely to be my only request for help from people
| on this list.

I actually suggested doing the exact amount of tests, but it differently by
not forcing Suggests: in.

I have the feeling you seem to think that I want to create more work not
less. I am not.


No, you are reading something into my messages that isn't there.  I 
think I understand your suggestion now (CRAN should test with no 
Suggested packages present, not with all of them present).  That is less 
work than they currently do.


I disagree with your suggestion for two reasons.

1. I think it would lead to less effective testing.

As Uwe said, tests would skip code that only runs when Suggested 
packages are present.  I think it would almost certainly lead to less 
code coverage in the tests.


2. It would be disruptive during the transition.

We could work around this by putting together a standard way for a 
package to indicate that some functionality is not available, without 
triggering an error, and the check code could ignore those messages when 
testing with missing Suggested packages.


Actually, I think I am the one who is suggesting the creation of more 
work.  In particular, I suggested you should do some work to address 
point 2 above.  I have also suggested that CRAN should test both with 
and without Suggested packages, which is obviously more work for the 
test machines, but also more work for whoever sets up the new testing code.


My hope is to spread the work out to package submitters, or people on 
this list, rather than on two or three people at CRAN.



I once thought I could influence the 'powers that be' to interpret the policy
they have written (ie Writing R Extensions) differently.  But I guess I was
wrong.


There are no powers, there are only overworked volunteers.

Duncan Murdoch




Dirk

| > | > No, CRAN could just flip that toggle and run once.
| > |
| > | I do not get it: It won't make much of a difference because I have most
| > | packages installed. It is only a diffrence for suggested packages that
| >
| > Which is a normal shortcut to make your life easier ... but not what other
| > systems do. Travis CI runs start from scratch with an empty base. So does
| > Debian for package build (and for subsequent tests such as the
| > reproducibility tests). In each case only listed packages get installed.
| >
| > When we let R do its business Suggests is (currently) interpreted as
| > Depends. I continue to claim that is not what it is for.
| >
| >

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

2016-08-03 Thread Dirk Eddelbuettel


On 3 August 2016 at 13:44, Duncan Murdoch wrote:
| On 03/08/2016 11:17 AM, Dirk Eddelbuettel wrote:
| >
| > On 3 August 2016 at 16:21, Uwe Ligges wrote:
| > |
| > |
| > | On 03.08.2016 16:14, Dirk Eddelbuettel wrote:
| > | >
| > | > On 3 August 2016 at 16:00, Uwe Ligges wrote:
| > | > | On 03.08.2016 14:24, Dirk Eddelbuettel wrote:
| > | > | > Then again, users of TravisCI know that just toggling
| > | > | >
| > | > | > _R_CHECK_FORCE_SUGGESTS_=FALSE
| > | > |
| > | > | I was travelling, hence a delayed response:
| > | > |
| > | > | Why users of TravisCI? It is documented in the manual. Setting it to
| > | >
| > | > Because Travis breaks your check as it works in a cleanroom with only 
the
| > | > specified packages installed. As it (and that is the gist of my 
argument)
| > | > should ...
| > |
| > | No, because if I suggest a package, I want to check also the code that
| > | uses a suggested package. So I have to have the package installed.
| >
| > But then you are treating Suggests as Depends and installing irregardless.
| 
| No, it is different from Depends.
| 
| Depends implies something will be attached to the search list.  Suggests 
| doesn't.  It is more like Imports, but without importing anything.

I am aware of the distinction.  I was talking, or trying to talk, about
'conditional' versus 'unconditional' "presence of packages" if you wish.

Depends/Import are the latter, Suggests is the former.

Or it would be if we chose to treat it like that. My gripe is that we don't.

| CRAN chooses to test in an environment where all Suggested packages are 
| available.  They could do more testing (and I am pretty strongly in 
| favour of that), e.g. *also* test in an environment where none of them 
| are available.  But lots of packages will fail those tests, hence my 
| request to you (not a gambit, I'm not playing a game) to estimate how 
| many, and work out how to fix those packages.

I did send a PR to one such package in the past. The package maintainers
ignored it.  So I stopped.

I think I'll do the same with this thread now. 
 
| Doing more testing will make it harder to get a package accepted on 
| CRAN, but hopefully will mean that if it makes it there, it will be of 
| better quality.
| 
| The other thing I am trying to do is to work out ways to spread out the 
| work that CRAN does to other people.  I believe the current model will 
| not last another year, and I'd rather that it didn't collapse 
| completely.  This is unlikely to be my only request for help from people 
| on this list.

I actually suggested doing the exact amount of tests, but it differently by
not forcing Suggests: in.

I have the feeling you seem to think that I want to create more work not
less. I am not.

I once thought I could influence the 'powers that be' to interpret the policy
they have written (ie Writing R Extensions) differently.  But I guess I was
wrong. 

Dirk
 
| > | > No, CRAN could just flip that toggle and run once.
| > |
| > | I do not get it: It won't make much of a difference because I have most
| > | packages installed. It is only a diffrence for suggested packages that
| >
| > Which is a normal shortcut to make your life easier ... but not what other
| > systems do. Travis CI runs start from scratch with an empty base. So does
| > Debian for package build (and for subsequent tests such as the
| > reproducibility tests). In each case only listed packages get installed.
| >
| > When we let R do its business Suggests is (currently) interpreted as
| > Depends. I continue to claim that is not what it is for.
| >
| > As Thomas pointed out, Suggests: seems to be overloaded already to two
| > distinct use cases.
| >
| > | are not available for my platform, and toggling it means checks always
| > | fail then.
| > |
| > |
| > | > What Duncan suggested was (as I understand it an empirically-driven
| > | > assessment of what it would take to get from here (set to FALSE, 
tolerate bad
| > | > Policy) to there (set to TRUE, IMHO better Policy adherence).
| > |
| > | You do not understand the environment variable, I believe.
| >
| > That is of course entirely possibly but it also seems that some of us in 
this
| > discussion are continuously talking past each other. I tried to make my case
| > but seemingly failed to explain it clearly enough.
| 
| That's the nature of the medium.  You need to be prepared to say the 
| same thing in different ways so that dense people like me understand them.
| 
| Duncan Murdoch
| 

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

2016-08-03 Thread Uwe Ligges




On 03.08.2016 17:17, Dirk Eddelbuettel wrote:


On 3 August 2016 at 16:21, Uwe Ligges wrote:
|
|
| On 03.08.2016 16:14, Dirk Eddelbuettel wrote:
| >
| > On 3 August 2016 at 16:00, Uwe Ligges wrote:
| > | On 03.08.2016 14:24, Dirk Eddelbuettel wrote:
| > | > Then again, users of TravisCI know that just toggling
| > | >
| > | > _R_CHECK_FORCE_SUGGESTS_=FALSE
| > |
| > | I was travelling, hence a delayed response:
| > |
| > | Why users of TravisCI? It is documented in the manual. Setting it to
| >
| > Because Travis breaks your check as it works in a cleanroom with only the
| > specified packages installed. As it (and that is the gist of my argument)
| > should ...
|
| No, because if I suggest a package, I want to check also the code that
| uses a suggested package. So I have to have the package installed.

But then you are treating Suggests as Depends and installing irregardless.


Yes, because I want to check all the runtime code on CRAN and not only 
subsets.





| > No, CRAN could just flip that toggle and run once.
|
| I do not get it: It won't make much of a difference because I have most
| packages installed. It is only a diffrence for suggested packages that

Which is a normal shortcut to make your life easier ... but not what other
systems do. Travis CI runs start from scratch with an empty base. So does
Debian for package build (and for subsequent tests such as the
reproducibility tests). In each case only listed packages get installed.


Yes, I'd install install time dependencies for the installtion 
procedures and all dependencies for the check part.




When we let R do its business Suggests is (currently) interpreted as
Depends.


No, as we check installtion without suggetsed packages.



I continue to claim that is not what it is for.

As Thomas pointed out, Suggests: seems to be overloaded already to two
distinct use cases.



Yes, and I know that a package should work without the suggestes 
installed, we do not disagree there.





| are not available for my platform, and toggling it means checks always
| fail then.
|
|
| > What Duncan suggested was (as I understand it an empirically-driven
| > assessment of what it would take to get from here (set to FALSE, tolerate 
bad
| > Policy) to there (set to TRUE, IMHO better Policy adherence).
|
| You do not understand the environment variable, I believe.

That is of course entirely possibly but it also seems that some of us in this
discussion are continuously talking past each other. I tried to make my case
but seemingly failed to explain it clearly enough.



So we agree that it would be nice to test with and without installed 
suggested packages? And the checks should give fine results in both 
cases, right?


Best,
Uwe

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

2016-08-03 Thread Uwe Ligges




On 03.08.2016 16:14, Dirk Eddelbuettel wrote:


On 3 August 2016 at 16:00, Uwe Ligges wrote:
| On 03.08.2016 14:24, Dirk Eddelbuettel wrote:
| > Then again, users of TravisCI know that just toggling
| >
| > _R_CHECK_FORCE_SUGGESTS_=FALSE
|
| I was travelling, hence a delayed response:
|
| Why users of TravisCI? It is documented in the manual. Setting it to

Because Travis breaks your check as it works in a cleanroom with only the
specified packages installed. As it (and that is the gist of my argument)
should ...


No, because if I suggest a package, I want to check also the code that 
uses a suggested package. So I have to have the package installed.





| true gives an error in the check if the suggested package is not
| available for the check. And typically you want to have it available to
| be able to actually check all parts of your package.
| So this is not what you are looking for.
|
| What you suggest means that we would have to run the checks twice, as
| Duncan explained already, once with and once without the suggetsed
| package installed.

No, CRAN could just flip that toggle and run once.


I do not get it: It won't make much of a difference because I have most 
packages installed. It is only a diffrence for suggested packages that 
are not available for my platform, and toggling it means checks always 
fail then.




What Duncan suggested was (as I understand it an empirically-driven
assessment of what it would take to get from here (set to FALSE, tolerate bad
Policy) to there (set to TRUE, IMHO better Policy adherence).


You do not understand the environment variable, I believe.

Uwe



Dirk



__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

2016-08-03 Thread Dirk Eddelbuettel


On 3 August 2016 at 16:00, Uwe Ligges wrote:
| On 03.08.2016 14:24, Dirk Eddelbuettel wrote:
| > Then again, users of TravisCI know that just toggling
| >
| > _R_CHECK_FORCE_SUGGESTS_=FALSE
| 
| I was travelling, hence a delayed response:
| 
| Why users of TravisCI? It is documented in the manual. Setting it to 

Because Travis breaks your check as it works in a cleanroom with only the
specified packages installed. As it (and that is the gist of my argument)
should ...

| true gives an error in the check if the suggested package is not 
| available for the check. And typically you want to have it available to 
| be able to actually check all parts of your package.
| So this is not what you are looking for.
| 
| What you suggest means that we would have to run the checks twice, as 
| Duncan explained already, once with and once without the suggetsed 
| package installed.

No, CRAN could just flip that toggle and run once.

What Duncan suggested was (as I understand it an empirically-driven
assessment of what it would take to get from here (set to FALSE, tolerate bad
Policy) to there (set to TRUE, IMHO better Policy adherence).

Dirk

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

2016-08-03 Thread Uwe Ligges




On 03.08.2016 14:24, Dirk Eddelbuettel wrote:


On 2 August 2016 at 19:45, Duncan Murdoch wrote:
| Why not put together code to implement your idea, and see how big the
| problem would be to phase it in, by seeing how many packages fail under it?

Ahh, the old 'put-up-or-shut-up' gambit, very nice. Big fan of that myself.

I have been sitting on the need to build better infrastructure for my reverse
depends checks anyway, and may get there over the fall.  Or not as I have
been saying that for a long time...  It will likely be Docker-based which may
or may not be suitable for CRAN, but should be for r-hub which may get this
all by itself too.  Not sure of this can be done in R alone.

Then again, users of TravisCI know that just toggling

_R_CHECK_FORCE_SUGGESTS_=FALSE


I was travelling, hence a delayed response:

Why users of TravisCI? It is documented in the manual. Setting it to 
true gives an error in the check if the suggested package is not 
available for the check. And typically you want to have it available to 
be able to actually check all parts of your package.

So this is not what you are looking for.

What you suggest means that we would have to run the checks twice, as 
Duncan explained already, once with and once without the suggetsed 
package installed.


Best,
Uwe








does it too so in that sense it is already there, but not used?

Dirk



__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

2016-08-03 Thread Uwe Ligges




On 03.08.2016 15:20, Thomas J. Leeper wrote:

The issue seems to boil down to the fact that Suggests is covering two
very different use cases: (a) conditional package code and (b)
example/test/vignette code.

Consider a package (say "foo") that is only used in tests. Users do
not need "foo" since package code never calls it. If our package
instead calls "foo" conditionally (requireNamespace(); foo::bar(),
etc.), then users may very well want "foo" and need it much more than
they would if "foo" were only used in tests. Yet DESCRIPTION does not
allow a distinction two be made between these two scenarios.

I think the length and complexity of the discussion in WRE[1] makes
clear that Suggests is being used two cover these two very different
use cases.



I am more worried about yet another dependency level that confuses more 
than it helps for the majority of developers and users.
As an example, I nneed to explain myself again and again what "Enhances" 
means (and perhaps I am still doing it wrong).


Best,
Uwe Ligges



-Thomas

[1] 
https://cran.r-project.org/doc/manuals/r-devel/R-exts.html#Package-Dependencies

Thomas J. Leeper
http://www.thomasleeper.com


On Wed, Aug 3, 2016 at 2:04 PM, Mark van der Loo
 wrote:

Recommends: only gets installed, can be used via if(requireNamespace())
from the package and in pkg tests[1] [snip]
Build-Depends: gets installed before build, removed after.
Suggests: only installed when requested at install.packages() and only
used in examples/vignettes.


[snip]

 I'd much rather
have a way of declaring explicitly the different aspects of dependence
on a package rather than bundling them up into cute labels,


Agreed


but it's too late for that now.  However, we don't need to make things
worse.


Disagreed. We could follow the well-established practices of Debian (and
CRAN already does that, partially).



If 'tons of packages' are using if(requireNamespace) in their package
code there seems to be a need for something like this. [snip]



I don't follow the argument here.  What problem are you solving?


Basically I'm trying to address the idea suggested by Thomas, who started
this conversation, and make it a bit more explicit. I felt that the
discussion went a little off-track there.

Right now, when package code (not examples) uses a suggested package, part
of that package will by default not work - at least that's how people use it
now. I would like it to work by default. For examples/vignettes you could be
more forgiving since running an example is not core functionality of a
package.


Perhaps more controversially a 'Breaks' field could be considered. [snip]



This isn't controversial, it's just a bad idea.  Don't encourage people
to break things.


Your reaction just proved my point about it being controversial. More
seriously, real progress is hardly ever possible without breaking things, so
I think at least people could have a serious discussion about it before
dismissing it simply as a bad idea. The Debian community obviously once
thought it was a good idea, so why not discuss it for R/CRAN? (discussions
are also an important way to progress even if no line of code is changed).
At the moment, I'm inclined against the idea, but I for one like to see me
proven wrong.



[1] actually, once we know a pkg is Recommended, the
'if(requireNamespace)' could even be absorbed in the :: operator.



I don't see how :: would be any different than it is now.  If you don't
have foo available, and you try to use foo::bar(), what would happen
other than an error?


I think you're right there. .

Best,
Mark


Op wo 3 aug. 2016 om 13:41 schreef Duncan Murdoch
:


On 03/08/2016 5:32 AM, Mark van der Loo wrote:


After reading the link in Dirk's initial reply, how about adding fields
'Recommends' and 'Build-Depends' to DESCRIPTION as in Debian?

Recommends: only gets installed, can be used via if(requireNamespace())
from the package and in pkg tests[1]. [Debian: The Recommends field
should list packages that would be found together with this one in all
but unusual installations.]
Build-Depends: gets installed before build, removed after.
Suggests: only installed when requested at install.packages() and only
used in examples/vignettes.


I think the distinction between Recommends and Suggests is too subtle
here.  I already think it's a bad thing that we are using these words in
ways that don't really correspond to English usage.  I'd much rather
have a way of declaring explicitly the different aspects of dependence
on a package rather than bundling them up into cute labels, but it's too
late for that now.  However, we don't need to make things worse.



If 'tons of packages' are using if(requireNamespace) in their package
code there seems to be a need for something like this. Compliance to the
above can be checked automatically and  a gradual implementation via
NOTE->WARNING->ERROR in R CMD check seems possible.


I don't follow the

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

2016-08-03 Thread Uwe Ligges




On 03.08.2016 15:04, Mark van der Loo wrote:

Recommends: only gets installed, can be used via if(requireNamespace())
from the package and in pkg tests[1] [snip]
Build-Depends: gets installed before build, removed after.
Suggests: only installed when requested at install.packages() and only
used in examples/vignettes.


[snip]

 I'd much rather
have a way of declaring explicitly the different aspects of dependence
on a package rather than bundling them up into cute labels,


Agreed


but it's too late for that now.  However, we don't need to make things

worse.

Disagreed. We could follow the well-established practices of Debian (and
CRAN already does that, partially).



If 'tons of packages' are using if(requireNamespace) in their package
code there seems to be a need for something like this. [snip]



I don't follow the argument here.  What problem are you solving?


Basically I'm trying to address the idea suggested by Thomas, who started
this conversation, and make it a bit more explicit. I felt that the
discussion went a little off-track there.

Right now, when package code (not examples) uses a suggested package, part
of that package will by default not work - at least that's how people use
it now. I would like it to work by default. For examples/vignettes you
could be more forgiving since running an example is not core functionality
of a package.


Perhaps more controversially a 'Breaks' field could be considered. [snip]



This isn't controversial, it's just a bad idea.  Don't encourage people
to break things.


Your reaction just proved my point about it being controversial. More
seriously, real progress is hardly ever possible without breaking things,
so I think at least people could have a serious discussion about it before
dismissing it simply as a bad idea. The Debian community obviously once
thought it was a good idea, so why not discuss it for R/CRAN? (discussions
are also an important way to progress even if no line of code is changed).
At the moment, I'm inclined against the idea, but I for one like to see me
proven wrong.



I don't see a point for the discussion, because there is even a CRAN 
policy that explain the procedure if there is really need (which should 
be carefully decided) to break things.



Best,
Uwe Ligges






[1] actually, once we know a pkg is Recommended, the
'if(requireNamespace)' could even be absorbed in the :: operator.



I don't see how :: would be any different than it is now.  If you don't
have foo available, and you try to use foo::bar(), what would happen
other than an error?


I think you're right there. .

Best,
Mark


Op wo 3 aug. 2016 om 13:41 schreef Duncan Murdoch WARNING->ERROR in R CMD check seems possible.


I don't follow the argument here.  What problem are you solving?


Perhaps more controversially a 'Breaks' field could be considered. There
are a few packages out there that have many, many, dependencies.
Implementing breaking updates currently depends on the willingness of
many authors to update their package or convincing the CRAN maintainers
to allow for (temporary) breakage.


This isn't controversial, it's just a bad idea.  Don't encourage people
to break things.


The suggestion to have functions auto-install things is very
inconvenient for the good reasons pointed out by Thomas. Additionally,
it is often based on the wrong assumptions. Example: the RGtk2 package
has this habit of trying to install when libgtk2 is not on the path. But
in my case that is often exactly the case: it is just not on the path
(libgtk2 is on the network, the VM just doesn't know yet). So I'd rather
have a proper and accurate error message (which is good practice anyway).


Best,
Mark

[1] actually, once we know a pkg is

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

2016-08-03 Thread Thomas J. Leeper

The issue seems to boil down to the fact that Suggests is covering two
very different use cases: (a) conditional package code and (b)
example/test/vignette code.

Consider a package (say "foo") that is only used in tests. Users do
not need "foo" since package code never calls it. If our package
instead calls "foo" conditionally (requireNamespace(); foo::bar(),
etc.), then users may very well want "foo" and need it much more than
they would if "foo" were only used in tests. Yet DESCRIPTION does not
allow a distinction two be made between these two scenarios.

I think the length and complexity of the discussion in WRE[1] makes
clear that Suggests is being used two cover these two very different
use cases.

-Thomas

[1] 
https://cran.r-project.org/doc/manuals/r-devel/R-exts.html#Package-Dependencies

Thomas J. Leeper
http://www.thomasleeper.com


On Wed, Aug 3, 2016 at 2:04 PM, Mark van der Loo
 wrote:
>>> Recommends: only gets installed, can be used via if(requireNamespace())
>>> from the package and in pkg tests[1] [snip]
>>> Build-Depends: gets installed before build, removed after.
>>> Suggests: only installed when requested at install.packages() and only
>>> used in examples/vignettes.
>
> [snip]
>>  I'd much rather
>> have a way of declaring explicitly the different aspects of dependence
>> on a package rather than bundling them up into cute labels,
>
> Agreed
>
>> but it's too late for that now.  However, we don't need to make things
>> worse.
>
> Disagreed. We could follow the well-established practices of Debian (and
> CRAN already does that, partially).
>
>
>>> If 'tons of packages' are using if(requireNamespace) in their package
>>> code there seems to be a need for something like this. [snip]
>
>> I don't follow the argument here.  What problem are you solving?
>
> Basically I'm trying to address the idea suggested by Thomas, who started
> this conversation, and make it a bit more explicit. I felt that the
> discussion went a little off-track there.
>
> Right now, when package code (not examples) uses a suggested package, part
> of that package will by default not work - at least that's how people use it
> now. I would like it to work by default. For examples/vignettes you could be
> more forgiving since running an example is not core functionality of a
> package.
>
>>> Perhaps more controversially a 'Breaks' field could be considered. [snip]
>
>> This isn't controversial, it's just a bad idea.  Don't encourage people
>> to break things.
>
> Your reaction just proved my point about it being controversial. More
> seriously, real progress is hardly ever possible without breaking things, so
> I think at least people could have a serious discussion about it before
> dismissing it simply as a bad idea. The Debian community obviously once
> thought it was a good idea, so why not discuss it for R/CRAN? (discussions
> are also an important way to progress even if no line of code is changed).
> At the moment, I'm inclined against the idea, but I for one like to see me
> proven wrong.
>
>
>>> [1] actually, once we know a pkg is Recommended, the
>>> 'if(requireNamespace)' could even be absorbed in the :: operator.
>
>>I don't see how :: would be any different than it is now.  If you don't
>>have foo available, and you try to use foo::bar(), what would happen
>>other than an error?
>
> I think you're right there. .
>
> Best,
> Mark
>
>
> Op wo 3 aug. 2016 om 13:41 schreef Duncan Murdoch
> :
>>
>> On 03/08/2016 5:32 AM, Mark van der Loo wrote:
>> >
>> > After reading the link in Dirk's initial reply, how about adding fields
>> > 'Recommends' and 'Build-Depends' to DESCRIPTION as in Debian?
>> >
>> > Recommends: only gets installed, can be used via if(requireNamespace())
>> > from the package and in pkg tests[1]. [Debian: The Recommends field
>> > should list packages that would be found together with this one in all
>> > but unusual installations.]
>> > Build-Depends: gets installed before build, removed after.
>> > Suggests: only installed when requested at install.packages() and only
>> > used in examples/vignettes.
>>
>> I think the distinction between Recommends and Suggests is too subtle
>> here.  I already think it's a bad thing that we are using these words in
>> ways that don't really correspond to English usage.  I'd much rather
>> have a way of declaring explicitly the different aspects of dependence
>> on a package rather than bundling them up into cute labels, but it's too
>> late for that now.  However, we don't need to make things worse.
>>
>> >
>> > If 'tons of packages' are using if(requireNamespace) in their package
>> > code there seems to be a need for something like this. Compliance to the
>> > above can be checked automatically and  a gradual implementation via
>> > NOTE->WARNING->ERROR in R CMD check seems possible.
>>
>> I don't follow the argument here.  What problem are you solving?
>>
>> > Perhaps more controversially a 'Breaks'

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

2016-08-03 Thread Dirk Eddelbuettel


On 2 August 2016 at 19:45, Duncan Murdoch wrote:
| Why not put together code to implement your idea, and see how big the
| problem would be to phase it in, by seeing how many packages fail under it?

Ahh, the old 'put-up-or-shut-up' gambit, very nice. Big fan of that myself.

I have been sitting on the need to build better infrastructure for my reverse
depends checks anyway, and may get there over the fall.  Or not as I have
been saying that for a long time...  It will likely be Docker-based which may
or may not be suitable for CRAN, but should be for r-hub which may get this
all by itself too.  Not sure of this can be done in R alone.

Then again, users of TravisCI know that just toggling

_R_CHECK_FORCE_SUGGESTS_=FALSE

does it too so in that sense it is already there, but not used?

Dirk

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

2016-08-03 Thread Duncan Murdoch

On 03/08/2016 5:32 AM, Mark van der Loo wrote:

After reading the link in Dirk's initial reply, how about adding fields
'Recommends' and 'Build-Depends' to DESCRIPTION as in Debian?

Recommends: only gets installed, can be used via if(requireNamespace())
from the package and in pkg tests[1]. [Debian: The Recommends field
should list packages that would be found together with this one in all
but unusual installations.]
Build-Depends: gets installed before build, removed after.
Suggests: only installed when requested at install.packages() and only
used in examples/vignettes.

I think the distinction between Recommends and Suggests is too subtle 
here.  I already think it's a bad thing that we are using these words in 
ways that don't really correspond to English usage.  I'd much rather 
have a way of declaring explicitly the different aspects of dependence 
on a package rather than bundling them up into cute labels, but it's too 
late for that now.  However, we don't need to make things worse.

If 'tons of packages' are using if(requireNamespace) in their package
code there seems to be a need for something like this. Compliance to the
above can be checked automatically and  a gradual implementation via
NOTE->WARNING->ERROR in R CMD check seems possible.

I don't follow the argument here.  What problem are you solving?

Perhaps more controversially a 'Breaks' field could be considered. There
are a few packages out there that have many, many, dependencies.
Implementing breaking updates currently depends on the willingness of
many authors to update their package or convincing the CRAN maintainers
to allow for (temporary) breakage.

This isn't controversial, it's just a bad idea.  Don't encourage people 
to break things.

The suggestion to have functions auto-install things is very
inconvenient for the good reasons pointed out by Thomas. Additionally,
it is often based on the wrong assumptions. Example: the RGtk2 package
has this habit of trying to install when libgtk2 is not on the path. But
in my case that is often exactly the case: it is just not on the path
(libgtk2 is on the network, the VM just doesn't know yet). So I'd rather
have a proper and accurate error message (which is good practice anyway).

Best,
Mark

[1] actually, once we know a pkg is Recommended, the
'if(requireNamespace)' could even be absorbed in the :: operator.

I don't see how :: would be any different than it is now.  If you don't 
have foo available, and you try to use foo::bar(), what would happen 
other than an error?

Duncan Murdoch

Op wo 3 aug. 2016 om 01:46 schreef Duncan Murdoch
>:

On 02/08/2016 6:34 PM, Dirk Eddelbuettel wrote:
>
> On 2 August 2016 at 18:13, Duncan Murdoch wrote:
> | Okay, now I think I understand, but I agree with CRAN.  It is not
> | feasible to tell if the test happened somewhere in the code
unless we
> | enforce a particular way of writing the test.
>
> Debian has well over 20k packages, and they are tested this way.
You just
> need to show the will of testing in an _empty_ environment to ensure
> _everything_ that is needed is loaded.
>
> | I would object if I had to write if (requireNamespace("foo"))
multiple
> | times just to satisfy CRAN's test, when any sane human could
tell that
> | the first test was sufficient.
> |
> | For example, if my package Suggests: foo, I should be able to write
> |
> | if (!requireNamespace("foo"))
> |stop("Package foo is needed for this example")
> |
> | and then merrily call foo::bar() as many times as I like.
> |
> | Or am I still misunderstanding you?  What particular thing
should CRAN
> | change?
>
> You seem to misunderstand that both you and I want
>
>   if (!requireNamespace("foo"))
>  stop("Package foo is needed for this example")
>
> (or alternative per-call tests) and that CRAN does not enforce either.
>
> CRAN, like Hadley, just closes its eyes, swallows hard, and then
simply loads
> everything treating Suggests as if it were Depends.
>
> But it ain't. Suggests != Depends.
>
> Now clearer?

So really what you're suggesting is that CRAN should run tests with the
suggested packages absent.  Presumably tests should also be run with
them present.

But if they did that, the code that I want to write would call stop()
and fail.  So we'd need some way to say "Let the user know they need
'foo' to run this, but don't fail."  And we'd need to phase this in
really gradually, because tons of packages are using code like my
example.

You volunteered to help CRAN package checking.  Why not put together
code to implement your idea, and see how big the problem would be to
phase it in, by seeing how many packages fail under it?

Duncan Murdoch

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

2016-08-03 Thread Mark van der Loo

After reading the link in Dirk's initial reply, how about adding
fields 'Recommends'
and 'Build-Depends' to DESCRIPTION as in Debian?

Recommends: only gets installed, can be used via if(requireNamespace())
from the package and in pkg tests[1]. [Debian: The Recommends field should
list packages that would be found together with this one in all but unusual
installations.]
Build-Depends: gets installed before build, removed after.
Suggests: only installed when requested at install.packages() and only used
in examples/vignettes.

If 'tons of packages' are using if(requireNamespace) in their package code
there seems to be a need for something like this. Compliance to the above
can be checked automatically and  a gradual implementation via
NOTE->WARNING->ERROR in R CMD check seems possible.

Perhaps more controversially a 'Breaks' field could be considered. There
are a few packages out there that have many, many, dependencies.
Implementing breaking updates currently depends on the willingness of many
authors to update their package or convincing the CRAN maintainers to allow
for (temporary) breakage.

The suggestion to have functions auto-install things is very inconvenient
for the good reasons pointed out by Thomas. Additionally, it is often based
on the wrong assumptions. Example: the RGtk2 package has this habit of
trying to install when libgtk2 is not on the path. But in my case that is
often exactly the case: it is just not on the path (libgtk2 is on the
network, the VM just doesn't know yet). So I'd rather have a proper and
accurate error message (which is good practice anyway).

Best,
Mark

[1] actually, once we know a pkg is Recommended, the 'if(requireNamespace)'
could even be absorbed in the :: operator.

Op wo 3 aug. 2016 om 01:46 schreef Duncan Murdoch :

> On 02/08/2016 6:34 PM, Dirk Eddelbuettel wrote:
> >
> > On 2 August 2016 at 18:13, Duncan Murdoch wrote:
> > | Okay, now I think I understand, but I agree with CRAN.  It is not
> > | feasible to tell if the test happened somewhere in the code unless we
> > | enforce a particular way of writing the test.
> >
> > Debian has well over 20k packages, and they are tested this way.  You
> just
> > need to show the will of testing in an _empty_ environment to ensure
> > _everything_ that is needed is loaded.
> >
> > | I would object if I had to write if (requireNamespace("foo")) multiple
> > | times just to satisfy CRAN's test, when any sane human could tell that
> > | the first test was sufficient.
> > |
> > | For example, if my package Suggests: foo, I should be able to write
> > |
> > | if (!requireNamespace("foo"))
> > |stop("Package foo is needed for this example")
> > |
> > | and then merrily call foo::bar() as many times as I like.
> > |
> > | Or am I still misunderstanding you?  What particular thing should CRAN
> > | change?
> >
> > You seem to misunderstand that both you and I want
> >
> >   if (!requireNamespace("foo"))
> >  stop("Package foo is needed for this example")
> >
> > (or alternative per-call tests) and that CRAN does not enforce either.
> >
> > CRAN, like Hadley, just closes its eyes, swallows hard, and then simply
> loads
> > everything treating Suggests as if it were Depends.
> >
> > But it ain't. Suggests != Depends.
> >
> > Now clearer?
>
>
> So really what you're suggesting is that CRAN should run tests with the
> suggested packages absent.  Presumably tests should also be run with
> them present.
>
> But if they did that, the code that I want to write would call stop()
> and fail.  So we'd need some way to say "Let the user know they need
> 'foo' to run this, but don't fail."  And we'd need to phase this in
> really gradually, because tons of packages are using code like my example.
>
> You volunteered to help CRAN package checking.  Why not put together
> code to implement your idea, and see how big the problem would be to
> phase it in, by seeing how many packages fail under it?
>
> Duncan Murdoch
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

2016-08-02 Thread Duncan Murdoch


On 02/08/2016 6:34 PM, Dirk Eddelbuettel wrote:


On 2 August 2016 at 18:13, Duncan Murdoch wrote:
| Okay, now I think I understand, but I agree with CRAN.  It is not
| feasible to tell if the test happened somewhere in the code unless we
| enforce a particular way of writing the test.

Debian has well over 20k packages, and they are tested this way.  You just
need to show the will of testing in an _empty_ environment to ensure
_everything_ that is needed is loaded.

| I would object if I had to write if (requireNamespace("foo")) multiple
| times just to satisfy CRAN's test, when any sane human could tell that
| the first test was sufficient.
|
| For example, if my package Suggests: foo, I should be able to write
|
| if (!requireNamespace("foo"))
|stop("Package foo is needed for this example")
|
| and then merrily call foo::bar() as many times as I like.
|
| Or am I still misunderstanding you?  What particular thing should CRAN
| change?

You seem to misunderstand that both you and I want

  if (!requireNamespace("foo"))
 stop("Package foo is needed for this example")

(or alternative per-call tests) and that CRAN does not enforce either.

CRAN, like Hadley, just closes its eyes, swallows hard, and then simply loads
everything treating Suggests as if it were Depends.

But it ain't. Suggests != Depends.

Now clearer?



So really what you're suggesting is that CRAN should run tests with the 
suggested packages absent.  Presumably tests should also be run with 
them present.


But if they did that, the code that I want to write would call stop() 
and fail.  So we'd need some way to say "Let the user know they need 
'foo' to run this, but don't fail."  And we'd need to phase this in 
really gradually, because tons of packages are using code like my example.


You volunteered to help CRAN package checking.  Why not put together 
code to implement your idea, and see how big the problem would be to 
phase it in, by seeing how many packages fail under it?


Duncan Murdoch

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

2016-08-02 Thread Duncan Murdoch


On 02/08/2016 4:20 PM, Dirk Eddelbuettel wrote:


On 2 August 2016 at 16:07, Duncan Murdoch wrote:
| On 02/08/2016 1:41 PM, Dirk Eddelbuettel wrote:
| >
| > On 2 August 2016 at 13:12, Duncan Murdoch wrote:
| > | On 02/08/2016 1:01 PM, Dirk Eddelbuettel wrote:
| > | >
| > | > On 2 August 2016 at 11:36, Joshua Ulrich wrote:
| > | > | Maybe I'm missing something, but isn't that the point of calling
| > | > | requireNamespace()?  For example:
| > | > | if (requireNamespace("suggestedPackage"))
| > | > |   stop("suggestedPackage required but not installed")
| > | > |
| > | > | That doesn't seem like a heavy burden for package writers when writing
| > | > | infrequently used functions in their package.  You could even add the
| > | > | install.packages command to the error message to instruct users to
| > | > | install the required package.
| > | >
| > | > [...]
| > | >
| > | > | I personally would not want install.packages() to install packages I'm
| > | > | unlikely to actually use.
| > | > |
| > | > | It's also not clear to me how importing rarely used functions causes
| > | > | bloat, but installing all packages for all rarely-used functions does
| > | > | not cause bloat.
| > | >
| > | > Sadly, some people whose advocacy is taken as religous gospel in some
| > | > circles, particularly beginners, advocate pretty much that: treat 
Suggests:
| > | > as Depends: and install it anyway because, hell, why would one tests.
| > | >
| > | > I regularly run (large) reverse depends checks against some of my more 
widely
| > | > used packages and run into this all the time.
| > | >
| > | > We (as a community, including CRAN whose gatekeepers I have failed to
| > | > convince about this on on multiple attempts) are doing this wrong.
| > | > "Suggests:" really means optionally, rather than unconditionally.  But 
it
| > | > would appear that you and I are about to only ones desiring that 
behaviour.
| > |
| > | I thought I understood Joshua's point (and agreed with it), but you also
| > | seem to be agreeing with him and I don't understand at all what you're
| > | saying.
| > |
| > | What is "this" in your last paragraph, that you have failed to convince
| > | CRAN gatekeepers about?
| >
| > It is really simple:  Having _both_ Suggests: foo _and_ an unconditonal call
| > to foo::bar() in the code.
| >
| > Whereas Josh and I argue that it needs to be conditional on requireNames()
| > coming back TRUE.
|
| I am feeling particularly dense today.  What about "Having _both_
| Suggests: foo _and_ an unconditonal call to foo::bar() in the code." did
| you fail to convince CRAN about?  That it is a good thing, or a bad thing?

It is a Really Bad Thing )TM) because I read Writing R Extensions as meaning
that these ought to be tested for presence (as Suggests != Depends, so we
cannot [rather: should not] assume them) yet package authors are sloppy (and
CRAN let's them, being charged as facilitator to a crime here) and BAM I get
a failing test as some bad, bad code unconditionally calls code that should
only be called conditional on a suggested package actuallt being there.


Okay, now I think I understand, but I agree with CRAN.  It is not 
feasible to tell if the test happened somewhere in the code unless we 
enforce a particular way of writing the test.


I would object if I had to write if (requireNamespace("foo")) multiple 
times just to satisfy CRAN's test, when any sane human could tell that 
the first test was sufficient.


For example, if my package Suggests: foo, I should be able to write

if (!requireNamespace("foo"))
  stop("Package foo is needed for this example")

and then merrily call foo::bar() as many times as I like.

Or am I still misunderstanding you?  What particular thing should CRAN 
change?


Duncan Murdoch

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

2016-08-02 Thread Dirk Eddelbuettel


On 2 August 2016 at 16:07, Duncan Murdoch wrote:
| On 02/08/2016 1:41 PM, Dirk Eddelbuettel wrote:
| >
| > On 2 August 2016 at 13:12, Duncan Murdoch wrote:
| > | On 02/08/2016 1:01 PM, Dirk Eddelbuettel wrote:
| > | >
| > | > On 2 August 2016 at 11:36, Joshua Ulrich wrote:
| > | > | Maybe I'm missing something, but isn't that the point of calling
| > | > | requireNamespace()?  For example:
| > | > | if (requireNamespace("suggestedPackage"))
| > | > |   stop("suggestedPackage required but not installed")
| > | > |
| > | > | That doesn't seem like a heavy burden for package writers when writing
| > | > | infrequently used functions in their package.  You could even add the
| > | > | install.packages command to the error message to instruct users to
| > | > | install the required package.
| > | >
| > | > [...]
| > | >
| > | > | I personally would not want install.packages() to install packages I'm
| > | > | unlikely to actually use.
| > | > |
| > | > | It's also not clear to me how importing rarely used functions causes
| > | > | bloat, but installing all packages for all rarely-used functions does
| > | > | not cause bloat.
| > | >
| > | > Sadly, some people whose advocacy is taken as religous gospel in some
| > | > circles, particularly beginners, advocate pretty much that: treat 
Suggests:
| > | > as Depends: and install it anyway because, hell, why would one tests.
| > | >
| > | > I regularly run (large) reverse depends checks against some of my more 
widely
| > | > used packages and run into this all the time.
| > | >
| > | > We (as a community, including CRAN whose gatekeepers I have failed to
| > | > convince about this on on multiple attempts) are doing this wrong.
| > | > "Suggests:" really means optionally, rather than unconditionally.  But 
it
| > | > would appear that you and I are about to only ones desiring that 
behaviour.
| > |
| > | I thought I understood Joshua's point (and agreed with it), but you also
| > | seem to be agreeing with him and I don't understand at all what you're
| > | saying.
| > |
| > | What is "this" in your last paragraph, that you have failed to convince
| > | CRAN gatekeepers about?
| >
| > It is really simple:  Having _both_ Suggests: foo _and_ an unconditonal call
| > to foo::bar() in the code.
| >
| > Whereas Josh and I argue that it needs to be conditional on requireNames()
| > coming back TRUE.
| 
| I am feeling particularly dense today.  What about "Having _both_ 
| Suggests: foo _and_ an unconditonal call to foo::bar() in the code." did 
| you fail to convince CRAN about?  That it is a good thing, or a bad thing?

It is a Really Bad Thing )TM) because I read Writing R Extensions as meaning
that these ought to be tested for presence (as Suggests != Depends, so we
cannot [rather: should not] assume them) yet package authors are sloppy (and
CRAN let's them, being charged as facilitator to a crime here) and BAM I get
a failing test as some bad, bad code unconditionally calls code that should
only be called conditional on a suggested package actuallt being there.

Hth,  Dirk

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

2016-08-02 Thread Duncan Murdoch


On 02/08/2016 1:41 PM, Dirk Eddelbuettel wrote:


On 2 August 2016 at 13:12, Duncan Murdoch wrote:
| On 02/08/2016 1:01 PM, Dirk Eddelbuettel wrote:
| >
| > On 2 August 2016 at 11:36, Joshua Ulrich wrote:
| > | Maybe I'm missing something, but isn't that the point of calling
| > | requireNamespace()?  For example:
| > | if (requireNamespace("suggestedPackage"))
| > |   stop("suggestedPackage required but not installed")
| > |
| > | That doesn't seem like a heavy burden for package writers when writing
| > | infrequently used functions in their package.  You could even add the
| > | install.packages command to the error message to instruct users to
| > | install the required package.
| >
| > [...]
| >
| > | I personally would not want install.packages() to install packages I'm
| > | unlikely to actually use.
| > |
| > | It's also not clear to me how importing rarely used functions causes
| > | bloat, but installing all packages for all rarely-used functions does
| > | not cause bloat.
| >
| > Sadly, some people whose advocacy is taken as religous gospel in some
| > circles, particularly beginners, advocate pretty much that: treat Suggests:
| > as Depends: and install it anyway because, hell, why would one tests.
| >
| > I regularly run (large) reverse depends checks against some of my more 
widely
| > used packages and run into this all the time.
| >
| > We (as a community, including CRAN whose gatekeepers I have failed to
| > convince about this on on multiple attempts) are doing this wrong.
| > "Suggests:" really means optionally, rather than unconditionally.  But it
| > would appear that you and I are about to only ones desiring that behaviour.
|
| I thought I understood Joshua's point (and agreed with it), but you also
| seem to be agreeing with him and I don't understand at all what you're
| saying.
|
| What is "this" in your last paragraph, that you have failed to convince
| CRAN gatekeepers about?

It is really simple:  Having _both_ Suggests: foo _and_ an unconditonal call
to foo::bar() in the code.

Whereas Josh and I argue that it needs to be conditional on requireNames()
coming back TRUE.


I am feeling particularly dense today.  What about "Having _both_ 
Suggests: foo _and_ an unconditonal call to foo::bar() in the code." did 
you fail to convince CRAN about?  That it is a good thing, or a bad thing?


Duncan Murdoch

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

2016-08-02 Thread Dirk Eddelbuettel


On 2 August 2016 at 13:12, Duncan Murdoch wrote:
| On 02/08/2016 1:01 PM, Dirk Eddelbuettel wrote:
| >
| > On 2 August 2016 at 11:36, Joshua Ulrich wrote:
| > | Maybe I'm missing something, but isn't that the point of calling
| > | requireNamespace()?  For example:
| > | if (requireNamespace("suggestedPackage"))
| > |   stop("suggestedPackage required but not installed")
| > |
| > | That doesn't seem like a heavy burden for package writers when writing
| > | infrequently used functions in their package.  You could even add the
| > | install.packages command to the error message to instruct users to
| > | install the required package.
| >
| > [...]
| >
| > | I personally would not want install.packages() to install packages I'm
| > | unlikely to actually use.
| > |
| > | It's also not clear to me how importing rarely used functions causes
| > | bloat, but installing all packages for all rarely-used functions does
| > | not cause bloat.
| >
| > Sadly, some people whose advocacy is taken as religous gospel in some
| > circles, particularly beginners, advocate pretty much that: treat Suggests:
| > as Depends: and install it anyway because, hell, why would one tests.
| >
| > I regularly run (large) reverse depends checks against some of my more 
widely
| > used packages and run into this all the time.
| >
| > We (as a community, including CRAN whose gatekeepers I have failed to
| > convince about this on on multiple attempts) are doing this wrong.
| > "Suggests:" really means optionally, rather than unconditionally.  But it
| > would appear that you and I are about to only ones desiring that behaviour.
| 
| I thought I understood Joshua's point (and agreed with it), but you also 
| seem to be agreeing with him and I don't understand at all what you're 
| saying.
| 
| What is "this" in your last paragraph, that you have failed to convince 
| CRAN gatekeepers about?

It is really simple:  Having _both_ Suggests: foo _and_ an unconditonal call
to foo::bar() in the code.

Whereas Josh and I argue that it needs to be conditional on requireNames()
coming back TRUE.

Dirk

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

2016-08-02 Thread Duncan Murdoch


On 02/08/2016 1:01 PM, Dirk Eddelbuettel wrote:


On 2 August 2016 at 11:36, Joshua Ulrich wrote:
| Maybe I'm missing something, but isn't that the point of calling
| requireNamespace()?  For example:
| if (requireNamespace("suggestedPackage"))
|   stop("suggestedPackage required but not installed")
|
| That doesn't seem like a heavy burden for package writers when writing
| infrequently used functions in their package.  You could even add the
| install.packages command to the error message to instruct users to
| install the required package.

[...]

| I personally would not want install.packages() to install packages I'm
| unlikely to actually use.
|
| It's also not clear to me how importing rarely used functions causes
| bloat, but installing all packages for all rarely-used functions does
| not cause bloat.

Sadly, some people whose advocacy is taken as religous gospel in some
circles, particularly beginners, advocate pretty much that: treat Suggests:
as Depends: and install it anyway because, hell, why would one tests.

I regularly run (large) reverse depends checks against some of my more widely
used packages and run into this all the time.

We (as a community, including CRAN whose gatekeepers I have failed to
convince about this on on multiple attempts) are doing this wrong.
"Suggests:" really means optionally, rather than unconditionally.  But it
would appear that you and I are about to only ones desiring that behaviour.


I thought I understood Joshua's point (and agreed with it), but you also 
seem to be agreeing with him and I don't understand at all what you're 
saying.


What is "this" in your last paragraph, that you have failed to convince 
CRAN gatekeepers about?


Duncan Murdoch

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

2016-08-02 Thread Dirk Eddelbuettel


On 2 August 2016 at 11:36, Joshua Ulrich wrote:
| Maybe I'm missing something, but isn't that the point of calling
| requireNamespace()?  For example:
| if (requireNamespace("suggestedPackage"))
|   stop("suggestedPackage required but not installed")
| 
| That doesn't seem like a heavy burden for package writers when writing
| infrequently used functions in their package.  You could even add the
| install.packages command to the error message to instruct users to
| install the required package.

[...]

| I personally would not want install.packages() to install packages I'm
| unlikely to actually use.
| 
| It's also not clear to me how importing rarely used functions causes
| bloat, but installing all packages for all rarely-used functions does
| not cause bloat.

Sadly, some people whose advocacy is taken as religous gospel in some
circles, particularly beginners, advocate pretty much that: treat Suggests:
as Depends: and install it anyway because, hell, why would one tests.

I regularly run (large) reverse depends checks against some of my more widely
used packages and run into this all the time.

We (as a community, including CRAN whose gatekeepers I have failed to
convince about this on on multiple attempts) are doing this wrong.
"Suggests:" really means optionally, rather than unconditionally.  But it
would appear that you and I are about to only ones desiring that behaviour.

Dirk

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

2016-08-02 Thread Thomas J. Leeper

Dirk,

That is useful. I think the "Recommends" category from that link is
pretty much what I am suggesting - something that should be installed
if it's available in the package repository but is not loaded/imported
by default.

Some of those other relationships sound like they would also be
useful, for example in the case you mention.

-Thomas

Thomas J. Leeper
http://www.thomasleeper.com

On Tue, Aug 2, 2016 at 5:34 PM, Dirk Eddelbuettel  wrote:
>
> See eg https://www.debian.org/doc/debian-policy/ch-relationships.html
>
> We have other big fish to fry.  Eg many packages have LinkingTo: BH (as Boost
> headers are indeed awesome) but this 120 mb (!!) payload is needed _once_
> during package compilation / installation and never again afterwards.  So
> Build-Depends: would be a nice addition too.
>
> Dirk
>
> --
> http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

2016-08-02 Thread Joshua Ulrich

On Tue, Aug 2, 2016 at 11:26 AM, Thomas J. Leeper  wrote:
> I have a fairly open-ended question about the handling of package
> dependencies that is inspired by the precise distinction between
> "Depends", "Imports", and "Suggests" listed in DESCRIPTION.
>
> Some background, as I understand it: Prior to requiring package
> namespaces, we listed package dependencies in "Depends" because the
> required package had to be (1) installed and (2) on the search path in
> order to access its functions. After introducing and requiring
> namespaces, there are very few circumstances in which "Depends" is
> needed because functions from the required package can either be
> imported (via "Imports" and NAMESPACE) and/or referred to using the
> `pkg::fun()` style. "Suggests" packages are maybe needed for examples,
> vignettes, etc. but are not installed by default via
> `install.packages()`, so are not guaranteed to be available.
>
> Some observations:
>
> 1. "Depends" seems to be less useful than before, except in rare cases
> where a package needs to be (a) installed, (b) loaded, and (c) on the
> search path. Imports covers most package dependency use cases.
>
> 2. There is a gap in functionality between "Imports" and "Suggests".
> Sometimes there is a need for functions that should be available
> (i.e., installed) but do not need to be loaded or imported because
> they are rarely used (e.g., graphing functions). Importing the
> functions adds bloat but only putting their package in "Suggests" does
> not guarantee availability when, for example, calling
> `requireNamespace()` or `require()`.
>
Maybe I'm missing something, but isn't that the point of calling
requireNamespace()?  For example:
if (requireNamespace("suggestedPackage"))
  stop("suggestedPackage required but not installed")

That doesn't seem like a heavy burden for package writers when writing
infrequently used functions in their package.  You could even add the
install.packages command to the error message to instruct users to
install the required package.

> Suggestion:
>
> Might it be useful to have a category between "Imports" and "Suggests"
> that covers packages that should be installed but not imported?
>
> This could be done by changing `install.packages()` to cover
> "Suggests" by default, changing the meaning of "Depends" and
> "Imports", or adding a new category to DESCRIPTION.
>
I personally would not want install.packages() to install packages I'm
unlikely to actually use.

It's also not clear to me how importing rarely used functions causes
bloat, but installing all packages for all rarely-used functions does
not cause bloat.

> I am interested in hearing your thoughts on this issue.
>
> Best,
> -Thomas
>
> Thomas J. Leeper
> http://www.thomasleeper.com
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel



-- 
Joshua Ulrich  |  about.me/joshuaulrich
FOSS Trading  |  www.fosstrading.com
R/Finance 2016 | www.rinfinance.com

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

2016-08-02 Thread Dirk Eddelbuettel


See eg https://www.debian.org/doc/debian-policy/ch-relationships.html

We have other big fish to fry.  Eg many packages have LinkingTo: BH (as Boost
headers are indeed awesome) but this 120 mb (!!) payload is needed _once_
during package compilation / installation and never again afterwards.  So
Build-Depends: would be a nice addition too.

Dirk

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies? - Part 2

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies? - Part 2

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies? - Part 2

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies? - Part 2

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies? - Part 2

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies? - Part 2

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies? - Part 2

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

Re: [R-pkg-devel] Handling Not-Always-Needed Dependencies?

30 matches

Site Navigation

Mail list logo

Footer information