Hi Laurent,
Here are the results of the experiment. As you can see below the issue
remains.
Thanks,
Wim
First lines of MY ORIGINALl python setup.py build:
[hpcapps@dirac 12072016]$ python setup.py build
R version 3.3.2 (2016-10-31) -- "Sincere Pumpkin Patch"
/uufs/chpc.utah.edu/sys/installdir/R/3.3.2bb/lib64/R/bin/R CMD config
--ldflags
/uufs/chpc.utah.edu/sys/installdir/R/3.3.2bb/lib64/R/bin/R CMD config
--cppflags
Compilation parameters for rpy2's C components:
include_dirs = ['/uufs/chpc.utah.edu/sys/
installdir/R/3.3.2bb/lib64/R/include']
library_dirs = ['/uufs/chpc.utah.edu/sys/
installdir/R/3.3.2bb/lib64/R/lib']
libraries = ['R', 'pcre', 'lzma', 'bz2', 'z', 'rt', 'dl',
'm', 'icuuc', 'icui18n']
extra_link_args = ['-Wl,--export-dynamic', '-fopenmp']
NEW COMPILATION:
setenv LD_LIBRARY_PATH /uufs/
chpc.utah.edu/sys/installdir/R/3.3.2bb/lib64/R/lib:/uufs/chpc.utah.edu/sys/installdir/openblas/0.2.19-st/lib
NOTE: that I added the paths manually to LD_LIBRARY_PATH
because
echo "`R RHOME`/`R CMD config LIBnn`"
/uufs/chpc.utah.edu/sys/installdir/R/3.3.2bb/lib64/R/lib64
The directory /uufs/chpc.utah.edu/sys/installdir/R/3.3.2bb/lib64/R/lib64
does not exist
cd /uufs/chpc.utah.edu/sys/srcdir/pylib/2.7.12bb/rpy2 ;
hg clone https://bitbucket.org/rpy2/rpy2 12072016
cd 12072016
python setup.py build
R version 3.3.2 (2016-10-31) -- "Sincere Pumpkin Patch"
/uufs/chpc.utah.edu/sys/installdir/R/3.3.2bb/lib64/R/bin/R CMD config
--ldflags
/uufs/chpc.utah.edu/sys/installdir/R/3.3.2bb/lib64/R/bin/R CMD config
--cppflags
Compilation parameters for rpy2's C components:
include_dirs = ['/uufs/
chpc.utah.edu/sys/installdir/R/3.3.2bb/lib64/R/include']
library_dirs = ['/uufs/
chpc.utah.edu/sys/installdir/R/3.3.2bb/lib64/R/lib']
libraries = ['R', 'pcre', 'lzma', 'bz2', 'z', 'rt', 'dl',
'm', 'icuuc', 'icui18n']
extra_link_args = ['-Wl,--export-dynamic', '-fopenmp']
python setup.py install --prefix=/uufs/
chpc.utah.edu/sys/installdir/python/2.7.12bb
python -m rpy2.tests -v
rpy2 version: 2.9.0dev
- built against R version: 3-3.2--71607
- running linked to R version: R version 3.3.2 (2016-10-31)
/uufs/
chpc.utah.edu/sys/installdir/python/2.7.12bb/lib/python2.7/site-packages/rpy2-2.9.0.dev0-py2.7-linux-x86_64.egg/rpy2/robjects/lib/ggplot2.py:67:
UserWarning: This was designed againt ggplot2 version 2.1.0 but you have
2.2.0
warnings.warn('This was designed againt ggplot2 version %s but you have
%s' % (TARGET_VERSION, ggplot2.__version__))
testCallErrorWhenEndedR (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
... ok
testConsolePrint (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ... ok
testExternalPython (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ...
ok
testExternalPythonFromExpression
(rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ... ok
testGet_initoptions (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ...
ok
testInitr (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ... ok
testInterruptR (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ... ok
testParse (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ... ok
testParseError (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ... ok
testParseIncompleteError
(rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ... ok
testParseInvalidString (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
... ok
testParseUnicode (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ... ok
testRpyMemory (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ...
Segmentation fault (core dumped)
As you can see there is no change.
On Wed, Dec 7, 2016 at 11:42 AM, Laurent Gautier <lgaut...@gmail.com> wrote:
> The command R is doing quite a bit of environment settings before starting
> the actual binary. Not all of this is done by rpy2 when starting, and I am
> guessing that it is trying to link / use the system's BLAS (and this is
> ending in a segfault).
>
> Try setting LD_LIBRARY_PATH to have both the path to your openblas library
> and to R's own shared library before running Python with rpy2.
> Something like this:
>
> export LD_LIBRARY_PATH=`R RHOME`/`R CMD config LIBnn`:/uufs/chpc.utah.edu/
> sys/installdir/openblas/0.2.19-st/lib
>
>
> L.
>
>
> 2016-12-07 13:27 GMT-05:00 Wim R. Cardoen <wcard...@gmail.com>:
>
>> Hi Laurent,
>>
>> Thanks for your reply.
>> These are the flags I used to compile R:
>> setenv CC gcc
>> setenv CXX g++
>> setenv F77 gfortran
>> setenv FC gfortran
>> setenv CFLAGS " -O2 -fPIC -I/uufs/chpc.utah.edu/sys/inst
>> alldir/openblas/0.2.19-st/include "
>> setenv CXXFLAGS " -O2 -fPIC -I/uufs/chpc.utah.edu/sys/inst
>> alldir/openblas/0.2.19-st/include "
>> setenv FFLAGS " -O2 -fPIC -I/uufs/chpc.utah.edu/sys/inst
>> alldir/openblas/0.2.19-st/include "
>> setenv FCFLAGS " -O2 -fPIC -I/uufs/chpc.utah.edu/sys/inst
>> alldir/openblas/0.2.19-st/include "
>> setenv LDFLAGS " -Wl,-rpath=/uufs/chpc.utah.edu
>> /sys/installdir/openblas/0.2.19-st/lib
>> -L/uufs/chpc.utah.edu/sys/inst
>> alldir/openblas/0.2.19-st/lib -lopenblas "
>>
>> # Build executables
>> cd /uufs/chpc.utah.edu/sys/builddir/R; mkdir 3.3.2bb; cd 3.3.2bb
>>
>> # Configure (with BLAS & LAPACK MKL Support)
>> ../../../srcdir/R/3.3.2/configure --prefix=/uufs/chpc.utah.edu/s
>> ys/installdir/R/3.3.2bb \
>> --enable-R-profiling --enable-R-shlib
>> --enable-memory-profiling \
>> --enable-java --enable-shared=yes
>> --with-blas="$LDFLAGS" \
>> --with-readline --with-cairo
>> --with-tcltk --with-libpng --with-jpeglib --with-libtiff \
>> --with-ICU --with-pic --with-x
>> --with-lapack
>>
>> ############################################################
>> ################################################
>> R is now configured for x86_64-pc-linux-gnu
>>
>> Source directory: ../../../srcdir/R/3.3.2
>> Installation directory: /uufs/chpc.utah.edu/sys/installdir/R/3.3.2bb
>>
>> C compiler: gcc -std=gnu99 -O2 -fPIC -I/uufs/
>> chpc.utah.edu/sys/installdir/openblas/0.2.19-st/include
>> Fortran 77 compiler: gfortran -O2 -fPIC -I/uufs/
>> chpc.utah.edu/sys/installdir/openblas/0.2.19-st/include
>>
>> C++ compiler: g++ -O2 -fPIC -I/uufs/
>> chpc.utah.edu/sys/installdir/openblas/0.2.19-st/include
>> C++11 compiler: g++ -std=c++11 -O2 -fPIC -I/uufs/
>> chpc.utah.edu/sys/installdir/openblas/0.2.19-st/include
>> Fortran 90/95 compiler: gfortran -O2 -fPIC -I/uufs/
>> chpc.utah.edu/sys/installdir/openblas/0.2.19-st/include
>> Obj-C compiler: gcc -g -O2 -fobjc-exceptions
>>
>> Interfaces supported: X11, tcltk
>> External libraries: readline, BLAS(OpenBLAS), LAPACK(in blas),
>> curl
>> Additional capabilities: PNG, JPEG, TIFF, NLS, cairo, ICU
>> Options enabled: shared R library, R profiling, memory
>> profiling
>>
>> Capabilities skipped:
>> Options not enabled: shared BLAS
>>
>> Recommended packages: yes
>>
>>
>> I have neither set LD_LIBRARY_PATH, nor $R_HOME but the R_LIBS_SITE env.
>> variable
>> echo $R_LIBS_SITE
>> /uufs/chpc.utah.edu/sys/installdir/RLibs/3.3.2bb
>> R CMD config BLAS_LIBS
>> -Wl,-rpath=/uufs/chpc.utah.edu/sys/installdir/openblas/0.2.19-st/lib
>> -L/uufs/chpc.utah.edu/sys/installdir/openblas/0.2.19-st/lib -lopenblas
>>
>> I installed a whole bunch of packages in the following way:
>> install.packages(pkgs=c("ggplot2","htmlwidgets","ggvis","
>> rgl","googleVis"),
>> lib=c("/uufs/chpc.utah.edu/sys/installdir/RLibs/3.3.2bb
>> "),
>> repos=c("http://cran.us.r-project.org";),
>> verbose=TRUE)
>>
>> hpcapps@dirac ~]$ R
>>
>> R version 3.3.2 (2016-10-31) -- "Sincere Pumpkin Patch"
>> Copyright (C) 2016 The R Foundation for Statistical Computing
>> Platform: x86_64-pc-linux-gnu (64-bit)
>>
>> R is free software and comes with ABSOLUTELY NO WARRANTY.
>> You are welcome to redistribute it under certain conditions.
>> Type 'license()' or 'licence()' for distribution details.
>>
>> Natural language support but running in an English locale
>>
>> R is a collaborative project with many contributors.
>> Type 'contributors()' for more information and
>> 'citation()' on how to cite R or R packages in publications.
>>
>> Type 'demo()' for some demos, 'help()' for on-line help, or
>> 'help.start()' for an HTML browser interface to help.
>> Type 'q()' to quit R.
>>
>> > .libPaths()
>> [1] "/uufs/chpc.utah.edu/sys/installdir/RLibs/3.3.2bb"
>> [2] "/uufs/chpc.utah.edu/sys/installdir/R/3.3.2bb/lib64/R/library"
>>
>>
>> PS:
>> Note that I have used the same (single threaded) openblas to compile
>> numpy.
>> I normally compile R with Intel MKL (threaded). The same is true for
>> python.
>> When I compiled R and numpy/scipy with the threaded Intel MKL libraries,
>> the tests of
>> RPy2 were hanging. That's why I went to the single threaded version of
>> BLAS/LAPACK.
>>
>>
>> Thanks,
>>
>> WIm
>>
>>
>>
>>
>> On Wed, Dec 7, 2016 at 10:43 AM, Laurent Gautier <lgaut...@gmail.com>
>> wrote:
>>
>>> Hi Wim,
>>>
>>> Segfaults can be hard to track without the ability to reproduce (not
>>> much you can do about for now, this seems something specific to your
>>> system).
>>>
>>> Do you have an environment variable R_HOME defined ? What are your
>>> LD_LIBRARY_PATH like ? Is your customized with ${R_HOME}/etc or ~/.R ? What
>>> does the shell command "R CMD config BLAS_LIBS" return ?
>>>
>>> L.
>>>
>>>
>>>
>>> 2016-12-07 12:12 GMT-05:00 Wim R. Cardoen <wcard...@gmail.com>:
>>>
>>>> Hello,
>>>>
>>>> I compiled R v 3.3.2 with openblas (single threaded version).
>>>> uname -a
>>>> >Linux dirac.chpc.utah.edu 3.10.0-327.36.3.el7.x86_64 #1 SMP Mon Oct
>>>> 24 16:09:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
>>>>
>>>> gcc:
>>>> gcc version 4.8.5 20150623 (Red Hat 4.8.5-4) (GCC)
>>>>
>>>> After installing the 2.9.0dev version of RPy2 (python 2.7.12) the tests
>>>> crashed:
>>>>
>>>> python -m rpy2.tests -v
>>>> rpy2 version: 2.9.0dev
>>>> - built against R version: 3-3.2--71607
>>>> - running linked to R version: R version 3.3.2 (2016-10-31)
>>>> /uufs/chpc.utah.edu/sys/installdir/python/2.7.12bb/lib/pytho
>>>> n2.7/site-packages/rpy2-2.9.0.dev0-py2.7-linux-x86_64.egg/rp
>>>> y2/robjects/lib/ggplot2.py:67: UserWarning: This was designed againt
>>>> ggplot2 version 2.1.0 but you have 2.2.0
>>>> warnings.warn('This was designed againt ggplot2 version %s but you
>>>> have %s' % (TARGET_VERSION, ggplot2.__version__))
>>>> testCallErrorWhenEndedR (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
>>>> ... ok
>>>> testConsolePrint (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
>>>> ... ok
>>>> testExternalPython (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
>>>> ... ok
>>>> testExternalPythonFromExpression
>>>> (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
>>>> ... ok
>>>> testGet_initoptions (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
>>>> ... ok
>>>> testInitr (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ... ok
>>>> testInterruptR (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ...
>>>> ok
>>>> testParse (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ... ok
>>>> testParseError (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ...
>>>> ok
>>>> testParseIncompleteError
>>>> (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
>>>> ... ok
>>>> testParseInvalidString (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
>>>> ... ok
>>>> testParseUnicode (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
>>>> ... ok
>>>> testRpyMemory (rinterface.tests.test_EmbeddedR.EmbeddedRTestCase) ...
>>>> Segmentation fault (core dumped)
>>>>
>>>> Any idea what went wrong? Please let me know if you need more info.
>>>>
>>>> Thanks,
>>>>
>>>> Wim
>>>>
>>>> ------------------------------------------------------------
>>>> ------------------
>>>> Developer Access Program for Intel Xeon Phi Processors
>>>> Access to Intel Xeon Phi processor-based developer platforms.
>>>> With one year of Intel Parallel Studio XE.
>>>> Training and support from Colfax.
>>>> Order your platform today.http://sdm.link/xeonphi
>>>> _______________________________________________
>>>> rpy-list mailing list
>>>> rpy-list@lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/rpy-list
>>>>
>>>>
>>>
>>
>>
>>
>>
>
------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/xeonphi
_______________________________________________
rpy-list mailing list
rpy-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rpy-list
