Re: [R-pkg-devel] Package valgrind problem I can't solve: Direction?

[Merlise Clyde, Ph.D.]

Peter,

I ran into similar problems with using valgrind to check  my package  BAS

From my understanding using valgrind with rhub does not run the same checks as

R -d "valgrind --tool=memcheck --leak-check=full" --vanilla < mypkg-Ex.R


see https://github.com/r-hub/rhub/issues/23
which might explain why it will run without errors there but the CRAN check 
will flag errors/warnings.  (perhaps someone else can confirm)


best,
Merlise


Merlise A Clyde
Professor & Chair Department of Statistical Science
Duke University
http://stat.duke.edu/~clyde

cl...@duke.edu
919 681 8440




On Nov 5, 2017, at 7:05 PM, Peter Dunn 
> wrote:

“unless I am being really stupid”

Lucky for that caveat. Feeling rather stupid for this late effort.

Thanks so much.  Help greatly appreciated.

P.



On 6/Nov/17, 9:51 am, "Iñaki Úcar" 
> wrote:

   2017-11-06 0:09 GMT+01:00 Peter Dunn 
>:
Impossible or not… it just happened (unless I am being really stupid, which
is entirely possible, indeed probable). I confirmed again this morning:
After rebuilding (R CMD build) and checking (R CMD check) without any
errors, I used rhub and the command line again:


Running valgrind at the command line, I get this error:

==5097== Conditional jump or move depends on uninitialised value(s)
==5097== at 0x1118F61FB: smallp_ (in
/Users/pdunn2/Library/R/3.4/library/tweedie/libs/tweedie.so)

   What I meant is that, if you run the script in tweedie.Rcheck from the
   command line, Rscript uses your *installed* version of tweedie, not
   the tarball you built with R CMD build. So if you didn't run R CMD
   INSTALL, Rscript is running an old version without any fix, hence the
   disagreement with R CMD check.

   Iñaki



USC, Locked Bag 4, Maroochydore DC, Queensland, 4558 Australia.
CRICOS Provider No: 01595D
Please consider the environment before printing this email.
This email is confidential. If received in error, please delete it from your 
system.

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_r-2Dpackage-2Ddevel=DwIGaQ=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc=NOkxkvdFOOffXzeTY2kgZQ=wQM2pJJEon1zRq8pImHXAnjSO5MT728HIQf-oNWJfAo=VTJwkGdmvI9lNBVHxBc_38mQ16_pBYZ3dLnKdZBvBwg=


[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Package valgrind problem I can't solve: Direction?

[Peter Dunn]

“unless I am being really stupid”

Lucky for that caveat. Feeling rather stupid for this late effort.

Thanks so much.  Help greatly appreciated.

P.



On 6/Nov/17, 9:51 am, "Iñaki Úcar"  wrote:

2017-11-06 0:09 GMT+01:00 Peter Dunn :
> Impossible or not… it just happened (unless I am being really stupid, 
which
> is entirely possible, indeed probable). I confirmed again this morning:
> After rebuilding (R CMD build) and checking (R CMD check) without any
> errors, I used rhub and the command line again:
>
>
> Running valgrind at the command line, I get this error:
>
> ==5097== Conditional jump or move depends on uninitialised value(s)
> ==5097== at 0x1118F61FB: smallp_ (in
> /Users/pdunn2/Library/R/3.4/library/tweedie/libs/tweedie.so)

What I meant is that, if you run the script in tweedie.Rcheck from the
command line, Rscript uses your *installed* version of tweedie, not
the tarball you built with R CMD build. So if you didn't run R CMD
INSTALL, Rscript is running an old version without any fix, hence the
disagreement with R CMD check.

Iñaki



USC, Locked Bag 4, Maroochydore DC, Queensland, 4558 Australia.
CRICOS Provider No: 01595D
Please consider the environment before printing this email.
This email is confidential. If received in error, please delete it from your 
system.

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Package valgrind problem I can't solve: Direction?

[Iñaki Úcar]

2017-11-06 0:09 GMT+01:00 Peter Dunn :
> Impossible or not… it just happened (unless I am being really stupid, which
> is entirely possible, indeed probable). I confirmed again this morning:
> After rebuilding (R CMD build) and checking (R CMD check) without any
> errors, I used rhub and the command line again:
>
>
> Running valgrind at the command line, I get this error:
>
> ==5097== Conditional jump or move depends on uninitialised value(s)
> ==5097== at 0x1118F61FB: smallp_ (in
> /Users/pdunn2/Library/R/3.4/library/tweedie/libs/tweedie.so)

What I meant is that, if you run the script in tweedie.Rcheck from the
command line, Rscript uses your *installed* version of tweedie, not
the tarball you built with R CMD build. So if you didn't run R CMD
INSTALL, Rscript is running an old version without any fix, hence the
disagreement with R CMD check.

Iñaki

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Package valgrind problem I can't solve: Direction?

[Peter Dunn]

Impossible or not… it just happened (unless I am being really stupid, which is 
entirely possible, indeed probable).  I confirmed again this morning: After 
rebuilding (R CMD build) and checking (R CMD check) without any errors, I used 
rhub and the command line again:


Running valgrind at the command line, I get this error:

==5097== Conditional jump or move depends on uninitialised value(s)
==5097==at 0x1118F61FB: smallp_ (in 
/Users/pdunn2/Library/R/3.4/library/tweedie/libs/tweedie.so)

…when I am here:

$ pwd
/Users/pdunn2/Documents/Research/Rpackages/tweedie/tweedie-debug/CRAN-update/tweedie.Rcheck



In rhub:

check_with_valgrind("tweedie")

gives:

── 0 errors ✔ | 0 warnings ✔ | 0 notes ✔
─  Done with R CMD check
─  Saving artifacts

…when I am here:

> getwd()
[1] 
"/Users/pdunn2/Documents/Research/Rpackages/tweedie/tweedie-debug/CRAN-update"




I will take this to mean that there really is no problem… and put this down to 
things I don’t understand and “experience”.

Thanks for everyone’s help.

P.

From: Iñaki Úcar <i.uca...@gmail.com>
Date: Friday, 3 November 2017 at 7:45 pm
To: Peter Dunn <pdu...@usc.edu.au>
Cc: "r-package-devel@r-project.org" <r-package-devel@r-project.org>
Subject: Re: [R-pkg-devel] Package valgrind problem I can't solve: Direction?

2017-11-03 6:01 GMT+01:00 Peter Dunn <pdu...@usc.edu.au>:
> Iñaki and all
>
> Well, thanks for pointers to rhub. Wonderful. Moving things to github, but
> have to go home now…
>
> So, when I download CRAN code, initialise w and lambda (which workled for
> Iñaki), and run
>
> rhub::check_with_valgrind()
>
> on the code, I get no errors
> (https://builder.r-hub.io/status/tweedie_2.2.5.tar.gz-c8873979fcf84b4f8a0a4d5a47175f63).
>
>
> But running
>
> R -d "valgrind --tool=memcheck --leak-check=full --track-origins=yes"
> --vanilla < tweedie-Ex.R
>
> from the command line *still* gives me errors about “Conditional jump or
> move depends on uninitialised value(s)” in the subroutine smallp”.

That's impossible. Did you rebuild and reinstall the package after
making those changes?

Iñaki



USC, Locked Bag 4, Maroochydore DC, Queensland, 4558 Australia.
CRICOS Provider No: 01595D
Please consider the environment before printing this email.
This email is confidential. If received in error, please delete it from your 
system.

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Package valgrind problem I can't solve: Direction?

[Iñaki Úcar]

2017-11-03 6:01 GMT+01:00 Peter Dunn :
> Iñaki and all
>
> Well, thanks for pointers to rhub. Wonderful. Moving things to github, but
> have to go home now…
>
> So, when I download CRAN code, initialise w and lambda (which workled for
> Iñaki), and run
>
> rhub::check_with_valgrind()
>
> on the code, I get no errors
> (https://builder.r-hub.io/status/tweedie_2.2.5.tar.gz-c8873979fcf84b4f8a0a4d5a47175f63).
>
>
> But running
>
> R -d "valgrind --tool=memcheck --leak-check=full --track-origins=yes"
> --vanilla < tweedie-Ex.R
>
> from the command line *still* gives me errors about “Conditional jump or
> move depends on uninitialised value(s)” in the subroutine smallp”.

That's impossible. Did you rebuild and reinstall the package after
making those changes?

Iñaki

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Package valgrind problem I can't solve: Direction?

[Peter Dunn]

Iñaki  and all

Well, thanks for pointers to rhub.  Wonderful.  Moving things to github, but 
have to go home now…

So, when I download CRAN code, initialise w and lambda (which workled for 
Iñaki), and run

rhub::check_with_valgrind()

on the code, I get no errors 
(https://builder.r-hub.io/status/tweedie_2.2.5.tar.gz-c8873979fcf84b4f8a0a4d5a47175f63).


But running

R -d "valgrind  --tool=memcheck --leak-check=full --track-origins=yes" 
--vanilla  < tweedie-Ex.R

from the command line *still* gives me errors about “Conditional jump or move 
depends on uninitialised value(s)” in the subroutine smallp”.

Iñaki found no errors when w and lambda were initialised.  I find similar using 
rhub (no errors)… but running valgrind from the command line still throws the 
same errors.

What does one make of that?

Thanks for any more help.


P.

PS: Really appreciate the help. This is driving me crazy.


From: Iñaki Úcar <i.uca...@gmail.com>
Date: Wednesday, 1 November 2017 at 10:28 pm
To: Peter Dunn <pdu...@usc.edu.au>
Cc: "r-package-devel@r-project.org" <r-package-devel@r-project.org>
Subject: Re: [R-pkg-devel] Package valgrind problem I can't solve: Direction?

2017-11-01 5:56 GMT+01:00 Peter Dunn <pdu...@usc.edu.au>:
> Wow: Thanks again. You are going above and beyond Iñaki which I appreciate
> greatly.
>
> But same error message appears, even after fixing (both the relerr, which I
> had already attended to, and the lambda issue).

All I can say is that initialising both lambda and w (from CRAN
version) solved the issue for me. As a matter of fact, I did see
another message about uninitialised values, but the back trace was
empty (no function names, I mean), which means that the problem is
somewhere else.

> I have since initialised *every* variable in the subroutine smallp (even the
> *input* variables to some arbitrary value to try to work out what is causing
> the issue) yet the error persists. There surely cannot be any uninitialized
> variables now *every* var in the subroutine is explicitly initialised.
>
> That tells me that there is nothing I can do to remove the error. If
> everything is initialised…
>
> Frustrated… And wasting so much time!

If you set up a public repo (GitHub or similar) where everyone can see
your current code base and changes made, it would be easier to help.
It would be even easier if you check your changes using a public build
system and share the results here. I don't know whether win-builder
uses valgrind, but you can use r-hub (https://github.com/r-hub/rhub)
by Gábor Csárdi. There is a function rhub::check_with_valgrind which
requires no further explanation.

Iñaki



USC, Locked Bag 4, Maroochydore DC, Queensland, 4558 Australia.
CRICOS Provider No: 01595D
Please consider the environment before printing this email.
This email is confidential. If received in error, please delete it from your 
system.

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Package valgrind problem I can't solve: Direction?

[Iñaki Úcar]

2017-11-01 5:56 GMT+01:00 Peter Dunn :
> Wow: Thanks again. You are going above and beyond Iñaki which I appreciate
> greatly.
>
> But same error message appears, even after fixing (both the relerr, which I
> had already attended to, and the lambda issue).

All I can say is that initialising both lambda and w (from CRAN
version) solved the issue for me. As a matter of fact, I did see
another message about uninitialised values, but the back trace was
empty (no function names, I mean), which means that the problem is
somewhere else.

> I have since initialised *every* variable in the subroutine smallp (even the
> *input* variables to some arbitrary value to try to work out what is causing
> the issue) yet the error persists. There surely cannot be any uninitialized
> variables now *every* var in the subroutine is explicitly initialised.
>
> That tells me that there is nothing I can do to remove the error. If
> everything is initialised…
>
> Frustrated… And wasting so much time!

If you set up a public repo (GitHub or similar) where everyone can see
your current code base and changes made, it would be easier to help.
It would be even easier if you check your changes using a public build
system and share the results here. I don't know whether win-builder
uses valgrind, but you can use r-hub (https://github.com/r-hub/rhub)
by Gábor Csárdi. There is a function rhub::check_with_valgrind which
requires no further explanation.

Iñaki

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Package valgrind problem I can't solve: Direction?

[Peter Dunn]

Wow: Thanks again.  You are going above and beyond Iñaki which I appreciate 
greatly.

But same error message appears, even after fixing (both the relerr, which I had 
already attended to, and the lambda issue).

I have since initialised *every* variable in the subroutine  smallp  (even the 
*input* variables to some arbitrary value to try to work out what is causing 
the issue) yet the error persists. There surely cannot be any uninitialized 
variables now *every* var in the subroutine is explicitly initialised.

That tells me that there is nothing I can do to remove the error. If everything 
is initialised…

Frustrated… And wasting so much time!

Thanks for any help from anyone.

P.

On 30/Oct/17, 9:06 pm, "Iñaki Úcar"  wrote:

2017-10-30 0:54 GMT+01:00 Peter Dunn :
> Thanks for this direction.
>
> But I am still confused, as the problem persists after attending to any
> possible issue: That piece of identified code only has four variables, 
and I
> can account for them all.
>
> The two variables its and aimrerr are now explicitly initialised. So they
> cannot be the problem.
>
> The variables maxit and relerr are inputs to the subroutine... and both 
are
> explicitly initialised in the only subroutine which calls smallp. So I 
can’t
> see how that can be a problem. (Clearly they cannot be initialised in the
> subroutine smallp or their input values are overwritten.)

Found it. "relerr" may be uninitialised because "w" is uninitialised
in smallp (tweedie 2.2.5, line 2169).

When you try to debug this kind of things, you must track not only
explicit initialisations, but also all the assignments with possibly
uninitialised variables.

BTW, I also see:

tweedie.f:335:0:

 if ( funvalue .LT. exp(-lambda) ) then

Warning: 'lambda' may be used uninitialized in this function
[-Wmaybe-uninitialized]

Regards,
Iñaki



USC, Locked Bag 4, Maroochydore DC, Queensland, 4558 Australia.
CRICOS Provider No: 01595D
Please consider the environment before printing this email.
This email is confidential. If received in error, please delete it from your 
system.

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Package valgrind problem I can't solve: Direction?

[Iñaki Úcar]

2017-10-30 0:54 GMT+01:00 Peter Dunn :
> Thanks for this direction.
>
> But I am still confused, as the problem persists after attending to any
> possible issue: That piece of identified code only has four variables, and I
> can account for them all.
>
> The two variables its and aimrerr are now explicitly initialised. So they
> cannot be the problem.
>
> The variables maxit and relerr are inputs to the subroutine... and both are
> explicitly initialised in the only subroutine which calls smallp. So I can’t
> see how that can be a problem. (Clearly they cannot be initialised in the
> subroutine smallp or their input values are overwritten.)

Found it. "relerr" may be uninitialised because "w" is uninitialised
in smallp (tweedie 2.2.5, line 2169).

When you try to debug this kind of things, you must track not only
explicit initialisations, but also all the assignments with possibly
uninitialised variables.

BTW, I also see:

tweedie.f:335:0:

 if ( funvalue .LT. exp(-lambda) ) then

Warning: 'lambda' may be used uninitialized in this function
[-Wmaybe-uninitialized]

Regards,
Iñaki

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Package valgrind problem I can't solve: Direction?

[Iñaki Úcar]

2017-10-30 0:54 GMT+01:00 Peter Dunn :
> Thanks for this direction.
>
> But I am still confused, as the problem persists after attending to any
> possible issue: That piece of identified code only has four variables, and I
> can account for them all.
>
> The two variables its and aimrerr are now explicitly initialised. So they
> cannot be the problem.
>
> The variables maxit and relerr are inputs to the subroutine... and both are
> explicitly initialised in the only subroutine which calls smallp. So I can’t
> see how that can be a problem. (Clearly they cannot be initialised in the
> subroutine smallp or their input values are overwritten.)

Maybe the location is wrong. What about the "result" variable in the
"smallp" subroutine? Shouldn't it be initialised?

> Any more idea for where I should be looking, or how I can progress this?
> Using the flags below when gfortran is called, with -O0 –g included,
> produced no errors or warnings at all.

Given that the issue disappears depending on the optimisation level,
maybe it is a false error.

Iñaki

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Package valgrind problem I can't solve: Direction?

[Iñaki Úcar]

Hi,

2017-10-26 5:16 GMT+02:00 Peter Dunn :
> Hi all
>
> I am trying to compile (on my Mac) an R package (tweedie) which includes 
> Fortran 77 code. I’m not much of a programmer, but can still manage to write 
> and update F77 code. I’m new to valgrind.
>
> In checking my package (which passes the build-and-check with no errors), I 
> use valgrind like this:
>
> R -d "valgrind  --tool=memcheck --leak-check=full --track-origins=yes" 
> --vanilla  < tweedie-Ex.R
>
>
> …on my package example, which gives me a series of messages like this:
>
> ==53843== Conditional jump or move depends on uninitialised value(s)
> ==53843==at 0x1118F61FB: smallp_ (in 
> /Users/pdunn2/Library/R/3.4/library/tweedie/libs/tweedie.so)
> ==53843==by 0xBFA4FB88FFEB: ???
> ==53843==by 0x3FF91CB2DA3A50E7: ???
> ==53843==by 0x104888757: ???
> ==53843==by 0xFFEC03E375AF: ???
> ==53843==by 0x3FE18C2EE1BE47CB: ???
> ==53843==by 0x104885DEF: ???
> ==53843==by 0x104885DF7: ???
> ==53843==by 0x3FE518F314C67BE7: ???
> ==53843==by 0x11364CEAF: ???
> ==53843==by 0x104885DBF: ???
> ==53843==by 0x10007: ??? (in 
> /Library/Frameworks/R.framework/Resources/bin/exec/R)
> ==53843==  Uninitialised value was created by a stack allocation
> ==53843==at 0x1118F59A6: smallp_ (in 
> /Users/pdunn2/Library/R/3.4/library/tweedie/libs/tweedie.so)

CRAN shows a complete backtrace:
https://www.stats.ox.ac.uk/pub/bdr/memtests/valgrind/tweedie/tweedie-Ex.Rout

According to it, the offending conditional jump seems to be this one:

 if ( ( its .LT. 3 )
 & .OR.
 &( ( its .LT. maxit ) .AND.
 &  ( abs(relerr) .GT. aimrerr )
 &)
 &  ) then

> I can see the piece of R code in the example from which this comes.  The 
> subroutine  smallp  is mentioned in the second line of the output.
>
> I simply cannot find the issue.  (Spent days looking so far.)  There are only 
> a few conditional jumps, and everything not passed to the subroutine  smallp  
> is initialised. I have checked the calls to  smallp  as well. I have compiled 
> the fortran code separately (after wrapping into a complete program), with 
> all kinds of flags:
>
> gfortran -O2 -fimplicit-none  -Wall  -Wline-truncation  
> -Wcharacter-truncation  -Wsurprising  -Waliasing  -Wunused-parameter  
> -fwhole-file  -fcheck=all  -std=f2008  -pedantic  -fbacktrace),

Did you try "-O0 -g" instead of "-O2"?

Regards,
Iñaki

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel