On 12-09-05 9:51 PM, Kasper Daniel Hansen wrote:
Duncan,
you are right that from the perspective of a single developer: it is
entirely possible to make tests conditional and ensure that 'long'
tests does not get run by CRAN.
What I, and several others, advocate is a uniform set of conventions,
so we avoid having multiple package specific solutions.
Martin has explained one reason for why using _R_CHECK_TIMINGS_ is not
ideal (the fact that we - at least I and Martin - are most interested
in the timings for the long tests).
You do realize that his objections were wrong, don't you?
Another reason is that I (and I
suspect most others) would prefer something that looks a bit more
official and supported.
The _R_CHECK_TIMINGS_ environment variable is documented and official.
High-jacking an environment variable set for
(what appears to be) a different reason does not seem like something
that could be considered supported long-term, and which have a uniform
interface (across packages).
Using it to detect reports about long-running examples is exactly what
it is designed for. It is not "high-jacking".
Of course, adding an extra environment variable or an extra function
(as suggested by Henrik) is additional work, and I know it is easy for
us to suggest things. My perspective is that I would like something
that will continue to be supported in the future, and where I can use
the same interface across the packages I maintain and others outside
my control that I occasionally test. Testing is not just being done
by the package author(s) or CRAN. I suspect many packages already
have half-baked solutions to the 'long' test vs 'short' test issue,
and I don't think we will see any movement toward a unified interface
without support in base R.
If Martin or I added a new environment variable _R_CHECK_LEVEL_, and
perhaps support code to let you set it in R CMD check or elsewhere, and
support functions to let you read it without the work of calling
Sys.getenv(), and documentation for its purpose -- that doesn't mean it
would be ever be used. I would have no objection whatsoever if he wants
to do that work, but I won't, as I think everyone who would ever use it
has already invented their own method for doing the same thing. It
would waste my time to implement it and would waste their time making
their code compatible with the new system.
And let me end by saying that I do agree with Martin: having numerical
levels is better (together with a clear statement what --as-cran sets
the level to). But I would rather have a binary state than nothing.
It was pointed out to me by a member of CRAN that detecting --as-cran is
insufficient. CRAN only runs incoming tests with settings equivalent to
that. The daily tests use different options. I don't know what those
are, and CRAN is unwilling to discuss their internal policies on this
list. (I would guess they find the tantrums and false claims here to be
frustrating. I do.)
So if Martin wants to implement something, I'd suggest that he
coordinate offline with CRAN.
Duncan Murdoch
Kasper
On Wed, Sep 5, 2012 at 9:10 AM, Duncan Murdoch <murdoch.dun...@gmail.com> wrote:
This post contains two incorrect statements. Since Martin is too busy to
post a retraction, I will point them out:
--timings and --as-cran do not set _R_CHECK_TIMINGS_ in the same way.
As far as I recall, nobody has suggested that package writers be limited to
two choices for test suites. I certainly haven't.
Duncan Murdoch
On 05/09/2012 6:25 AM, Martin Maechler wrote:
Deepayan Sarkar <deepayan.sar...@gmail.com>
on Wed, 5 Sep 2012 11:49:37 +0530 writes:
> On Wed, Sep 5, 2012 at 11:41 AM, Duncan Murdoch
> <murdoch.dun...@gmail.com> wrote:
>> On 12-09-04 8:19 PM, Kasper Daniel Hansen wrote:
>>>
>>> On Tue, Sep 4, 2012 at 6:02 PM, Duncan Murdoch
<murdoch.dun...@gmail.com>
>>> wrote:
>>>>
>>>> On 04/09/2012 5:42 PM, Dirk Eddelbuettel wrote:
>>>>>
>>>>>
>>>>> On 4 September 2012 at 17:26, Duncan Murdoch wrote:
>>>>> | On 04/09/2012 5:14 PM, Dirk Eddelbuettel wrote:
>>>>> | > An add-on argument to the already established option
--as-cran may
>>>>> be
>>>>> the
>>>>> | > best.
>>>>> | >
>>>>> | > And to iterate, what bugs me is that for _me_ on _my_
machine
>>>>> developing _my_
>>>>> | > package I have remember how to enable what is now (as per
CRAN's
>>>>> decree)
>>>>> | > "non-standard behaviour" of full testing. I fully agree
with what
>>>>> Terry had
>>>>> | > said: more tests are better (when we develop). I want the
full
>>>>> suite
>>>>> at my
>>>>> | > end; that is after all why we wrote it!
>>>>> |
>>>>> | You don't have to remember that, you need to figure it out
once, write
>>>>> a
>>>>> | script that sets the environment variables that enable it,
and then
>>>>> you
>>>>> | can forget it.
>>>>>
>>>>> "In theory, theory and practice are the same. In practice, they
are
>>>>> not."
>>>>>
>>>>> The main test script long had exactly such a setting; I wrote
what I
>>>>> wrote
>>>>> because it is _still the wrong way around_ and as I happen to
have added
>>>>> to
>>>>> unit tests this weekend _having suffered through precisely this
>>>>> setting_.
>>>>>
>>>>> But we are on different wavelengths here and I evidently do not
get my
>>>>> point
>>>>> across to you. And as you are the one who could make a change
where it
>>>>> matters, I have no choice but to rest my case in frustration.
>>>>
>>>>
>>>>
>>>> If you want to give up, then give up, but then don't complain
about the
>>>> current behaviour. If you want to fix it, then continue the
discussion.
>>>>
>>>> You're right that we're on different wavelengths. If you want
some tests
>>>> to
>>>> run at home but not on CRAN, then somewhere there has to be a
>>>> conditional.
>>>> I'm suggesting that the conditional should be "if there's a
tight time
>>>> limit, skip this".
>>>>
>>>> I don't remember if this was your suggestion, but someone has
suggested
>>>> "if
>>>> we're running with the --as-cran option, skip this" and others
have
>>>> suggested "if we're running on CRAN, skip this". I don't see
why you
>>>> find
>>>> my suggestion so objectionable. If you want, I'll repeat why I
find the
>>>> other two suggestions objectionable.
>>>
>>>
>>> I agree with Duncan that having an option long/short makes more
sense
>>> than with/without cran, as long as cran sets that option to be
short.
>>
>>>
>>>
>>> I would also prefer a command line switch to R CMD check to an
>>> environment variable, but I'll be very happy with a standardized
>>> environment variable.
>>
>>
>> I honestly don't see the need for a standardized variable. I've
told you
>> how to detect that you are running --as-cran; if that isn't
sufficient
>> information, then you, the package author, need to set up
something more
>> elaborate, and assume that if it's not set up, then someone else
(maybe
>> CRAN) is running the test.
> So maybe documenting that (_R_CHECK_TIMINGS_) more formally in
R-exts
> would be sufficient?
> -Deepayan
yes and no.
As Duncan said very early, --as-cran is just turning on several
already existing options, one of them being the
_R_CHECK_TIMINGS_ -- which you can *also* enable by --timings.
So, checking for _R_CHECK_TIMINGS_ is *not* checking for
the presence of --as-cran !
.. in the sense that it is necessary but not sufficient.
Also, if I run the "long checks" I may still want to use
--timings. For me as package developer, the timings may be even more
important in the "long" than in the "short" checks case.
So, one solution that is little work would be that --as-cran
sets an *extra* environment variable that is *only* set by
--as-cran, but by no other command line switch.
Still a pity that it seems people want to live in a 0/1
setting when it would be very natural to adopt a 0/1/2/3
(or so) one.
It's a bit like prefering verbose = FALSE/TRUE as argument to
an R function where eventually I often found that using a
tracing = 0/1/2 (or then 'verbose' = 0/1/2 which is almost
back compatible to FALSE/TRUE)
was more useful.
>> I asked the CRAN powers-that-be about the possibility of querying
the amount
>> of time remaining before a timeout; since the different platforms
all use
>> different mechanisms to enforce a timeout, that's not really
practical. So
>> the best you could hope for is to know that a timeout is in
effect. Before
>> I wrote any code, I'd need to hear why --as-cran detection isn't
sufficient.
I agree that *reliable* --as-cran detection solves the OP's
and most other correspondents problem.
But as argued above, _R_CHECK_TIMINGS_ is not sufficient.
Martin
______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
