> The directory /uufs/chpc.utah.edu/sys/installdir/R/3.3.2bb/lib64/R/lib64
does not exist
Do you end up with the path to the R shared library somewhere in
LD_LIBRARY_PATH ?
Did you check the expected libraries are loaded. For example with:
strace python -m rpy2.tests
2016-12-07 14:56 GMT-05:00 Wim R. Cardoen <wcard...@gmail.com>:
> Hi Laurent,
>
> Here are the results of the experiment. As you can see below the issue
> remains.
>
> Thanks,
>
> Wim
>
> First lines of MY ORIGINALl python setup.py build:
> [hpcapps@dirac 12072016]$ python setup.py build
> R version 3.3.2 (2016-10-31) -- "Sincere Pumpkin Patch"
> /uufs/chpc.utah.edu/sys/installdir/R/3.3.2bb/lib64/R/bin/R CMD config
> --ldflags
> /uufs/chpc.utah.edu/sys/installdir/R/3.3.2bb/lib64/R/bin/R CMD config
> --cppflags
>
> Compilation parameters for rpy2's C components:
> include_dirs = ['/uufs/chpc.utah.edu/sys/inst
> alldir/R/3.3.2bb/lib64/R/include']
> library_dirs = ['/uufs/chpc.utah.edu/sys/inst
> alldir/R/3.3.2bb/lib64/R/lib']
> libraries = ['R', 'pcre', 'lzma', 'bz2', 'z', 'rt', 'dl',
> 'm', 'icuuc', 'icui18n']
> extra_link_args = ['-Wl,--export-dynamic', '-fopenmp']
>
>
> NEW COMPILATION:
> setenv LD_LIBRARY_PATH /uufs/chpc.utah.edu/sys/
> installdir/R/3.3.2bb/lib64/R/lib:/uufs/chpc.utah.edu/sys/
> installdir/openblas/0.2.19-st/lib
>
> NOTE: that I added the paths manually to LD_LIBRARY_PATH
> because
> echo "`R RHOME`/`R CMD config LIBnn`"
> /uufs/chpc.utah.edu/sys/installdir/R/3.3.2bb/lib64/R/lib64
>
> The directory /uufs/chpc.utah.edu/sys/installdir/R/3.3.2bb/lib64/R/lib64
> does not exist
>
> cd /uufs/chpc.utah.edu/sys/srcdir/pylib/2.7.12bb/rpy2 ;
> hg clone https://bitbucket.org/rpy2/rpy2 12072016
> cd 12072016
> python setup.py build
> R version 3.3.2 (2016-10-31) -- "Sincere Pumpkin Patch"
> /uufs/chpc.utah.edu/sys/installdir/R/3.3.2bb/lib64/R/bin/R CMD config
> --ldflags
> /uufs/chpc.utah.edu/sys/installdir/R/3.3.2bb/lib64/R/bin/R CMD config
> --cppflags
>
> Compilation parameters for rpy2's C components:
> include_dirs = ['/uufs/chpc.utah.edu/sys/
> installdir/R/3.3.2bb/lib64/R/include']
> library_dirs = ['/uufs/chpc.utah.edu/sys/
> installdir/R/3.3.2bb/lib64/R/lib']
> libraries = ['R', 'pcre', 'lzma', 'bz2', 'z', 'rt', 'dl',
> 'm', 'icuuc', 'icui18n']
> extra_link_args = ['-Wl,--export-dynamic', '-fopenmp']
>
> python setup.py install --prefix=/uufs/chpc.utah.edu/
> sys/installdir/python/2.7.12bb
>
> python -m rpy2.tests -v
> rpy2 version: 2.9.0dev
> - built against R version: 3-3.2--71607
> - running linked to R version: R version 3.3.2 (2016-10-31)
> /uufs/chpc.utah.edu/sys/installdir/python/2.7.12bb/
> lib/python2.7/site-packages/rpy2-2.9.0.dev0-py2.7-linux-
> x86_64.egg/rpy2/robjects/lib/ggplot2.py:67: UserWarning: This was
> designed againt ggplot2 version 2.1.0 but you have 2.2.0
> warnings.warn('This was designed againt ggplot2 version %s but you have
> %s' % (TARGET_VERSION, ggplot2.__version__))
> testCallErrorWhenEndedR (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
> ... ok
> testConsolePrint (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ...
> ok
> testExternalPython (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
> ... ok
> testExternalPythonFromExpression
> (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
> ... ok
> testGet_initoptions (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
> ... ok
> testInitr (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ... ok
> testInterruptR (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ... ok
> testParse (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ... ok
> testParseError (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ... ok
> testParseIncompleteError (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
> ... ok
> testParseInvalidString (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
> ... ok
> testParseUnicode (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ...
> ok
> testRpyMemory (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ...
> Segmentation fault (core dumped)
>
> As you can see there is no change.
>
>
>
> On Wed, Dec 7, 2016 at 11:42 AM, Laurent Gautier <lgaut...@gmail.com>
> wrote:
>
>> The command R is doing quite a bit of environment settings before
>> starting the actual binary. Not all of this is done by rpy2 when starting,
>> and I am guessing that it is trying to link / use the system's BLAS (and
>> this is ending in a segfault).
>>
>> Try setting LD_LIBRARY_PATH to have both the path to your openblas
>> library and to R's own shared library before running Python with rpy2.
>> Something like this:
>>
>> export LD_LIBRARY_PATH=`R RHOME`/`R CMD config LIBnn`:/uufs/
>> chpc.utah.edu/sys/installdir/openblas/0.2.19-st/lib
>>
>>
>> L.
>>
>>
>> 2016-12-07 13:27 GMT-05:00 Wim R. Cardoen <wcard...@gmail.com>:
>>
>>> Hi Laurent,
>>>
>>> Thanks for your reply.
>>> These are the flags I used to compile R:
>>> setenv CC gcc
>>> setenv CXX g++
>>> setenv F77 gfortran
>>> setenv FC gfortran
>>> setenv CFLAGS " -O2 -fPIC -I/uufs/chpc.utah.edu/sys/inst
>>> alldir/openblas/0.2.19-st/include "
>>> setenv CXXFLAGS " -O2 -fPIC -I/uufs/chpc.utah.edu/sys/inst
>>> alldir/openblas/0.2.19-st/include "
>>> setenv FFLAGS " -O2 -fPIC -I/uufs/chpc.utah.edu/sys/inst
>>> alldir/openblas/0.2.19-st/include "
>>> setenv FCFLAGS " -O2 -fPIC -I/uufs/chpc.utah.edu/sys/inst
>>> alldir/openblas/0.2.19-st/include "
>>> setenv LDFLAGS " -Wl,-rpath=/uufs/chpc.utah.edu
>>> /sys/installdir/openblas/0.2.19-st/lib
>>> -L/uufs/chpc.utah.edu/sys/inst
>>> alldir/openblas/0.2.19-st/lib -lopenblas "
>>>
>>> # Build executables
>>> cd /uufs/chpc.utah.edu/sys/builddir/R; mkdir 3.3.2bb; cd 3.3.2bb
>>>
>>> # Configure (with BLAS & LAPACK MKL Support)
>>> ../../../srcdir/R/3.3.2/configure --prefix=/uufs/chpc.utah.edu/s
>>> ys/installdir/R/3.3.2bb \
>>> --enable-R-profiling --enable-R-shlib
>>> --enable-memory-profiling \
>>> --enable-java --enable-shared=yes
>>> --with-blas="$LDFLAGS" \
>>> --with-readline --with-cairo
>>> --with-tcltk --with-libpng --with-jpeglib --with-libtiff \
>>> --with-ICU --with-pic --with-x
>>> --with-lapack
>>>
>>> ############################################################
>>> ################################################
>>> R is now configured for x86_64-pc-linux-gnu
>>>
>>> Source directory: ../../../srcdir/R/3.3.2
>>> Installation directory: /uufs/chpc.utah.edu/sys/insta
>>> lldir/R/3.3.2bb
>>>
>>> C compiler: gcc -std=gnu99 -O2 -fPIC -I/uufs/
>>> chpc.utah.edu/sys/installdir/openblas/0.2.19-st/include
>>> Fortran 77 compiler: gfortran -O2 -fPIC -I/uufs/
>>> chpc.utah.edu/sys/installdir/openblas/0.2.19-st/include
>>>
>>> C++ compiler: g++ -O2 -fPIC -I/uufs/
>>> chpc.utah.edu/sys/installdir/openblas/0.2.19-st/include
>>> C++11 compiler: g++ -std=c++11 -O2 -fPIC -I/uufs/
>>> chpc.utah.edu/sys/installdir/openblas/0.2.19-st/include
>>> Fortran 90/95 compiler: gfortran -O2 -fPIC -I/uufs/
>>> chpc.utah.edu/sys/installdir/openblas/0.2.19-st/include
>>> Obj-C compiler: gcc -g -O2 -fobjc-exceptions
>>>
>>> Interfaces supported: X11, tcltk
>>> External libraries: readline, BLAS(OpenBLAS), LAPACK(in blas),
>>> curl
>>> Additional capabilities: PNG, JPEG, TIFF, NLS, cairo, ICU
>>> Options enabled: shared R library, R profiling, memory
>>> profiling
>>>
>>> Capabilities skipped:
>>> Options not enabled: shared BLAS
>>>
>>> Recommended packages: yes
>>>
>>>
>>> I have neither set LD_LIBRARY_PATH, nor $R_HOME but the R_LIBS_SITE env.
>>> variable
>>> echo $R_LIBS_SITE
>>> /uufs/chpc.utah.edu/sys/installdir/RLibs/3.3.2bb
>>> R CMD config BLAS_LIBS
>>> -Wl,-rpath=/uufs/chpc.utah.edu/sys/installdir/openblas/0.2.19-st/lib
>>> -L/uufs/chpc.utah.edu/sys/installdir/openblas/0.2.19-st/lib -lopenblas
>>>
>>> I installed a whole bunch of packages in the following way:
>>> install.packages(pkgs=c("ggplot2","htmlwidgets","ggvis","rgl
>>> ","googleVis"),
>>> lib=c("/uufs/chpc.utah.edu/sys/installdir/RLibs/3.3.2bb
>>> "),
>>> repos=c("http://cran.us.r-project.org";),
>>> verbose=TRUE)
>>>
>>> hpcapps@dirac ~]$ R
>>>
>>> R version 3.3.2 (2016-10-31) -- "Sincere Pumpkin Patch"
>>> Copyright (C) 2016 The R Foundation for Statistical Computing
>>> Platform: x86_64-pc-linux-gnu (64-bit)
>>>
>>> R is free software and comes with ABSOLUTELY NO WARRANTY.
>>> You are welcome to redistribute it under certain conditions.
>>> Type 'license()' or 'licence()' for distribution details.
>>>
>>> Natural language support but running in an English locale
>>>
>>> R is a collaborative project with many contributors.
>>> Type 'contributors()' for more information and
>>> 'citation()' on how to cite R or R packages in publications.
>>>
>>> Type 'demo()' for some demos, 'help()' for on-line help, or
>>> 'help.start()' for an HTML browser interface to help.
>>> Type 'q()' to quit R.
>>>
>>> > .libPaths()
>>> [1] "/uufs/chpc.utah.edu/sys/installdir/RLibs/3.3.2bb"
>>> [2] "/uufs/chpc.utah.edu/sys/installdir/R/3.3.2bb/lib64/R/library"
>>>
>>>
>>> PS:
>>> Note that I have used the same (single threaded) openblas to compile
>>> numpy.
>>> I normally compile R with Intel MKL (threaded). The same is true for
>>> python.
>>> When I compiled R and numpy/scipy with the threaded Intel MKL libraries,
>>> the tests of
>>> RPy2 were hanging. That's why I went to the single threaded version of
>>> BLAS/LAPACK.
>>>
>>>
>>> Thanks,
>>>
>>> WIm
>>>
>>>
>>>
>>>
>>> On Wed, Dec 7, 2016 at 10:43 AM, Laurent Gautier <lgaut...@gmail.com>
>>> wrote:
>>>
>>>> Hi Wim,
>>>>
>>>> Segfaults can be hard to track without the ability to reproduce (not
>>>> much you can do about for now, this seems something specific to your
>>>> system).
>>>>
>>>> Do you have an environment variable R_HOME defined ? What are your
>>>> LD_LIBRARY_PATH like ? Is your customized with ${R_HOME}/etc or ~/.R ? What
>>>> does the shell command "R CMD config BLAS_LIBS" return ?
>>>>
>>>> L.
>>>>
>>>>
>>>>
>>>> 2016-12-07 12:12 GMT-05:00 Wim R. Cardoen <wcard...@gmail.com>:
>>>>
>>>>> Hello,
>>>>>
>>>>> I compiled R v 3.3.2 with openblas (single threaded version).
>>>>> uname -a
>>>>> >Linux dirac.chpc.utah.edu 3.10.0-327.36.3.el7.x86_64 #1 SMP Mon Oct
>>>>> 24 16:09:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
>>>>>
>>>>> gcc:
>>>>> gcc version 4.8.5 20150623 (Red Hat 4.8.5-4) (GCC)
>>>>>
>>>>> After installing the 2.9.0dev version of RPy2 (python 2.7.12) the
>>>>> tests crashed:
>>>>>
>>>>> python -m rpy2.tests -v
>>>>> rpy2 version: 2.9.0dev
>>>>> - built against R version: 3-3.2--71607
>>>>> - running linked to R version: R version 3.3.2 (2016-10-31)
>>>>> /uufs/chpc.utah.edu/sys/installdir/python/2.7.12bb/lib/pytho
>>>>> n2.7/site-packages/rpy2-2.9.0.dev0-py2.7-linux-x86_64.egg/rp
>>>>> y2/robjects/lib/ggplot2.py:67: UserWarning: This was designed againt
>>>>> ggplot2 version 2.1.0 but you have 2.2.0
>>>>> warnings.warn('This was designed againt ggplot2 version %s but you
>>>>> have %s' % (TARGET_VERSION, ggplot2.__version__))
>>>>> testCallErrorWhenEndedR
>>>>> (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
>>>>> ... ok
>>>>> testConsolePrint (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
>>>>> ... ok
>>>>> testExternalPython (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
>>>>> ... ok
>>>>> testExternalPythonFromExpression
>>>>> (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
>>>>> ... ok
>>>>> testGet_initoptions (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
>>>>> ... ok
>>>>> testInitr (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ... ok
>>>>> testInterruptR (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
>>>>> ... ok
>>>>> testParse (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ... ok
>>>>> testParseError (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
>>>>> ... ok
>>>>> testParseIncompleteError
>>>>> (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
>>>>> ... ok
>>>>> testParseInvalidString (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
>>>>> ... ok
>>>>> testParseUnicode (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
>>>>> ... ok
>>>>> testRpyMemory (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ...
>>>>> Segmentation fault (core dumped)
>>>>>
>>>>> Any idea what went wrong? Please let me know if you need more info.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Wim
>>>>>
>>>>> ------------------------------------------------------------
>>>>> ------------------
>>>>> Developer Access Program for Intel Xeon Phi Processors
>>>>> Access to Intel Xeon Phi processor-based developer platforms.
>>>>> With one year of Intel Parallel Studio XE.
>>>>> Training and support from Colfax.
>>>>> Order your platform today.http://sdm.link/xeonphi
>>>>> _______________________________________________
>>>>> rpy-list mailing list
>>>>> rpy-list@lists.sourceforge.net
>>>>> https://lists.sourceforge.net/lists/listinfo/rpy-list
>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>>
>>
>
>
>
>
------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/xeonphi
_______________________________________________
rpy-list mailing list
rpy-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rpy-list
