Using the viennacl-omp profile I'm getting an out of memory error even with
Xmx set in MAHOUT_OPTS to 1g:

- mmul microbenchmark
  + Mahout multiplication time: 8920 ms.
  + ViennaCL/cpu/OpenMP multiplication time: 616 ms.
- trans
[INFO] Creating org.apache.mahout.viennacl.opencl.GPUMMul solver
[WARN] Unable to create class GPUMMul: attempting OpenMP version
[INFO] Creating org.apache.mahout.viennacl.openmp.OMPMMul solver
[INFO] Unable to create class OMPMMul: falling back to java version
*** RUN ABORTED ***
  java.lang.OutOfMemoryError: Java heap space
  at
it.unimi.dsi.fastutil.ints.Int2DoubleOpenHashMap.rehash(Int2DoubleOpenHashMap.java:1059)
  at
it.unimi.dsi.fastutil.ints.Int2DoubleOpenHashMap.insert(Int2DoubleOpenHashMap.java:295)
  at
it.unimi.dsi.fastutil.ints.Int2DoubleOpenHashMap.put(Int2DoubleOpenHashMap.java:301)
  at
org.apache.mahout.math.RandomAccessSparseVector.setQuick(RandomAccessSparseVector.java:130)
  at
org.apache.mahout.math.SparseRowMatrix.setQuick(SparseRowMatrix.java:105)
  at org.apache.mahout.math.AbstractMatrix.assign(AbstractMatrix.java:256)
  at
org.apache.mahout.math.scalabindings.MatrixOps.$colon$eq(MatrixOps.scala:192)
  at
org.apache.mahout.math.scalabindings.MatrixOps.cloned(MatrixOps.scala:260)
  at
org.apache.mahout.math.scalabindings.MatrixOps.$minus(MatrixOps.scala:66)
  at
org.apache.mahout.viennacl.openmp.ViennaCLSuiteOMP$$anonfun$4.apply$mcV$sp(ViennaCLSuiteOMP.scala:152)
  ...
[INFO]
------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Mahout Build Tools ................................. SUCCESS [
 1.131 s]
[INFO] Apache Mahout ...................................... SUCCESS [
 0.023 s]
[INFO] Mahout Math ........................................ SUCCESS [
31.439 s]
[INFO] Mahout HDFS ........................................ SUCCESS [
 2.151 s]
[INFO] Mahout Math Scala bindings ......................... SUCCESS [02:30
min]
[INFO] Mahout Spark bindings .............................. SUCCESS [
43.884 s]
[INFO] Mahout Flink bindings .............................. SUCCESS [04:13
min]
[INFO] Mahout Spark bindings shell ........................ SUCCESS [
 4.071 s]
[INFO] Mahout Release Package ............................. SUCCESS [
 1.311 s]
[INFO] Mahout H2O backend ................................. SUCCESS [02:50
min]
[INFO] Mahout Native VienniaCL OpenMP Bindings ............ FAILURE [
36.106 s]

On Sun, Oct 16, 2016 at 9:43 PM, Trevor Grant <trevor.d.gr...@gmail.com>
wrote:

> NOTE:
>
> The last solution creates a cyclical dependency-
>
> The way I made it work was happen-stance.
>
> 1)
> mvn clean install -Pviennacl -Phadoop2 -DskipTests
>
> 2)
> update the pom as described previously
>
> 3)
> cd math-scala
>
> 4)
> mvn test
>
> that will make it work- hacky but POC.
>
> Next how to do this 'right'.
>
> tg
>
>
>
> Trevor Grant
> Data Scientist
> https://github.com/rawkintrevo
> http://stackexchange.com/users/3002022/rawkintrevo
> http://trevorgrant.org
>
> *"Fortunate is he, who is able to know the causes of things."  -Virgil*
>
>
> On Sun, Oct 16, 2016 at 11:29 PM, Trevor Grant <trevor.d.gr...@gmail.com>
> wrote:
>
> > Maybe not the long term fix, but adding
> >
> > <dependency>
> >      <groupId>org.apache.mahout</groupId>
> >      <artifactId>mahout-native-viennacl_${scala.compat.
> > version}</artifactId>
> >      <scope>test</scope>
> >    </dependency>
> >
> > to math-scala/pom.xml seems to fix the afore mentioned errors
> >
> >
> > Trevor Grant
> > Data Scientist
> > https://github.com/rawkintrevo
> > http://stackexchange.com/users/3002022/rawkintrevo
> > http://trevorgrant.org
> >
> > *"Fortunate is he, who is able to know the causes of things."  -Virgil*
> >
> >
> > On Sun, Oct 16, 2016 at 10:23 PM, Andrew Palumbo <ap....@outlook.com>
> > wrote:
> >
> >> Thanks Trevor and Andrew- I think that both of your environments are
> >> fine.. (Ubuntu 16.04 was just to recommended because ViennaCL 1.7.1 is
> >> easily installed with sudo-apt-get install.  As logn as you have 1.7.1
> on
> >> the correct path, you should be fine.
> >>
> >>
> >> I'm getting these errors now also, and by viewing the stack trace, I can
> >> see that the mahout_native_vienniacl*.jars did not make it onto the
> >> classpath some how:
> >>
> >>
> >> [INFO] Creating org.apache.mahout.viennacl.opencl.GPUMMul solver
> >> [WARN] Unable to create class GPUMMul: attempting OpenMP version
> >> java.lang.ClassNotFoundException: org.apache.mahout.viennacl.ope
> >> ncl.GPUMMul$
> >>     at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> >>     at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> >>     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> >>     at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> >>     at java.lang.Class.forName0(Native Method)
> >>     at java.lang.Class.forName(Class.java:264)
> >>     at org.apache.mahout.math.scalabindings.SolverFactory$.getOpera
> >> tor(SolverFactory.scala:50)
> >>
> >>
> >> I think that i might have merged some conflicts badly when I rebased
> last
> >> week, because I was not seeing these errors.  I'll tey to get this
> worked
> >> out in the next couple of days.
> >>
> >>
> >>
> >> ________________________________
> >> From: Trevor Grant <trevor.d.gr...@gmail.com>
> >> Sent: Sunday, October 16, 2016 11:04:46 PM
> >> To: dev@mahout.apache.org
> >> Subject: Re: Trying out PR 261
> >>
> >> Having similar issues-
> >>
> >> Running 14.04 (I know, supposed to be 16.04, but whatever bear with me)
> >>
> >> Downloaded viennacl source, unzipped tar-dir/viennacl to
> >> /usr/include/viennacl (e.g. 'installed' viennacl 1.7.1)
> >>
> >> Updated gcc (though I don't think this was neccessary, now that I see
> >> Andrew M was having the same issues)
> >> sudo add-apt-repository ppa:ubuntu-toolchain-r/test
> >> sudo apt-get update
> >> sudo apt-get install gcc-5 g++-5
> >>
> >> sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-5 0
> >> --slave /usr/bin/g++ g++ /usr/bin/g++-5
> >>
> >>
> >> System / GPU info
> >> +------------------------------------------------------+
> >>
> >> | NVIDIA-SMI 352.63     Driver Version: 352.63         |
> >>
> >> |-------------------------------+----------------------+----
> >> ------------------+
> >> | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile
> Uncorr.
> >> ECC |
> >> | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util
> Compute
> >> M. |
> >> |===============================+======================+====
> >> ==================|
> >> |   0  GeForce GT 740      Off  | 0000:02:00.0     N/A |
> >>  N/A |
> >> | 33%   34C    P8    N/A /  N/A |    411MiB /  1021MiB |     N/A
> >>  Default |
> >> +-------------------------------+----------------------+----
> >> ------------------+
> >>
> >>
> >>
> >>
> >> Build was successful- no problems there.
> >>
> >> Still getting a slew of:
> >>
> >> [32mMatlabLikeMatrixOpsSuite: [0m
> >> [32m- multiplication [0m
> >> [32m- Hadamard [0m
> >> [32mRLikeVectorOpsSuite: [0m
> >> [32m- Hadamard [0m
> >> 19489 [ScalaTest-main-running-RLikeVectorOpsSuite] DEBUG
> >> org.apache.mahout.math.scalabindings.RLikeVectorOpsSuite  - dense
> vector
> >> dots:127.00 ms.
> >> 19489 [ScalaTest-main-running-RLikeVectorOpsSuite] DEBUG
> >> org.apache.mahout.math.scalabindings.RLikeVectorOpsSuite  - dense view
> >> dots:201.00 ms.
> >> [32m- dot-view performance [0m
> >> [INFO] Creating org.apache.mahout.viennacl.opencl.GPUMMul solver
> >> [WARN] Unable to create class GPUMMul: attempting OpenMP version
> >> [INFO] Creating org.apache.mahout.viennacl.openmp.OMPMMul solver
> >> [INFO] Unable to create class OMPMMul: falling back to java version
> >> 19491 [ScalaTest-main-running-MathSuite] DEBUG
> >> org.apache.mahout.math.scalabindings.MMul$  - AAt computation detected.
> >> [INFO] Creating org.apache.mahout.viennacl.opencl.GPUMMul solver
> >> [WARN] Unable to create class GPUMMul: attempting OpenMP version
> >> [INFO] Creating org.apache.mahout.viennacl.openmp.OMPMMul solver
> >> [INFO] Unable to create class OMPMMul: falling back to java version
> >> [INFO] Creating org.apache.mahout.viennacl.opencl.GPUMMul solver
> >> [WARN] Unable to create class GPUMMul: attempting OpenMP version
> >> [INFO] Creating org.apache.mahout.viennacl.openmp.OMPMMul solver
> >> [INFO] Unable to create class OMPMMul: falling back to java version
> >> ... goes on like that for a long long time
> >>
> >>
> >>
> >> Trevor Grant
> >> Data Scientist
> >> https://github.com/rawkintrevo
> >> http://stackexchange.com/users/3002022/rawkintrevo
> >> http://trevorgrant.org
> >>
> >> *"Fortunate is he, who is able to know the causes of things."  -Virgil*
> >>
> >>
> >> On Sun, Oct 16, 2016 at 6:59 PM, Andrew Musselman <
> >> andrew.mussel...@gmail.com> wrote:
> >>
> >> > I'm not sure my setup is correct; on Ubuntu 16.04 I'm getting a
> >> firehose of
> >> > infos and warns about not finding the GPUMMul and OMPMMul classes.
> I've
> >> > installed libviennacl-dev 1.7.1 through apt-get. Any ideas?
> >> >
> >> > Here's repro:
> >> > $ git fetch origin pull/261/head:viennacl-opmmul-a
> >> > $ mvn clean install -Pviennacl -Phadoop2
> >> >
> >> > Infos and warns (these are littered consistently throughout test
> logs):
> >> > %*% B mapBlock {}
> >> > - C = A %*% B incompatible B keys
> >> > [INFO] Creating org.apache.mahout.viennacl.opencl.GPUMMul solver
> >> > [WARN] Unable to create class GPUMMul: attempting OpenMP version
> >> > [INFO] Creating org.apache.mahout.viennacl.openmp.OMPMMul solver
> >> > [INFO] Unable to create class OMPMMul: falling back to java version
> >> > - Spark-specific C = At %*% B , join
> >> > [INFO] Creating org.apache.mahout.viennacl.opencl.GPUMMul solver
> >> > [WARN] Unable to create class GPUMMul: attempting OpenMP version
> >> > [INFO] Creating org.apache.mahout.viennacl.openmp.OMPMMul solver
> >> > [INFO] Unable to create class OMPMMul: falling back to java version
> >> > - C = At %*% B , join, String-keyed
> >> > [INFO] Creating org.apache.mahout.viennacl.opencl.GPUMMul solver
> >> > [WARN] Unable to create class GPUMMul: attempting OpenMP version
> >> > [INFO] Creating org.apache.mahout.viennacl.openmp.OMPMMul solver
> >> > [INFO] Unable to create class OMPMMul: falling back to java version
> >> > - C = At %*% B , zippable, String-keyed
> >> > [INFO] Creating org.apache.mahout.viennacl.opencl.GPUMMul solver
> >> > [WARN] Unable to create class GPUMMul: attempting OpenMP version
> >> > [INFO] Creating org.apache.mahout.viennacl.openmp.OMPMMul solver
> >> > [INFO] Unable to create class OMPMMul: falling back to java version
> >> > - C = A %*% B.t
> >> > [INFO] Creating org.apache.mahout.viennacl.opencl.GPUMMul solver
> >> > [WARN] Unable to create class GPUMMul: attempting OpenMP version
> >> > [INFO] Creating org.apache.mahout.viennacl.openmp.OMPMMul solver
> >> > [INFO] Unable to create class OMPMMul: falling back to java version
> >> > {
> >> >  0 => {0:26.0,1:35.0,2:46.0,3:51.0}
> >> >  1 => {0:50.0,1:69.0,2:92.0,3:105.0}
> >> >  2 => {0:62.0,1:86.0,2:115.0,3:132.0}
> >> >  3 => {0:74.0,1:103.0,2:138.0,3:159.0}
> >> > }
> >> > - C = A %*% inCoreB
> >> > [INFO] Creating org.apache.mahout.viennacl.opencl.GPUMMul solver
> >> > [WARN] Unable to create class GPUMMul: attempting OpenMP version
> >> > [INFO] Creating org.apache.mahout.viennacl.openmp.OMPMMul solver
> >> > [INFO] Unable to create class OMPMMul: falling back to java version
> >> > {
> >> >  0 => {0:26.0,1:35.0,2:46.0,3:51.0}
> >> >  1 => {0:50.0,1:69.0,2:92.0,3:105.0}
> >> >  2 => {0:62.0,1:86.0,2:115.0,3:132.0}
> >> >  3 => {0:74.0,1:103.0,2:138.0,3:159.0}
> >> > }
> >> >
> >> > Output of `clinfo`:
> >> > $ clinfo
> >> > Number of platforms                               1
> >> >   Platform Name                                   NVIDIA CUDA
> >> >   Platform Vendor                                 NVIDIA Corporation
> >> >   Platform Version                                OpenCL 1.2 CUDA
> 8.0.0
> >> >   Platform Profile                                FULL_PROFILE
> >> >   Platform Extensions
> >> > cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics
> >> > cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics
> >> > cl_khr_fp64 cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing
> >> > cl_nv_compiler_options cl_nv_device_attribute_query
> cl_nv_pragma_unroll
> >> > cl_nv_copy_opts
> >> >   Platform Extensions function suffix             NV
> >> >
> >> >   Platform Name                                   NVIDIA CUDA
> >> > Number of devices                                 1
> >> >   Device Name                                     GeForce GTX 750 Ti
> >> >   Device Vendor                                   NVIDIA Corporation
> >> >   Device Vendor ID                                0x10de
> >> >   Device Version                                  OpenCL 1.2 CUDA
> >> >   Driver Version                                  367.44
> >> >   Device OpenCL C Version                         OpenCL C 1.2
> >> >   Device Type                                     GPU
> >> >   Device Profile                                  FULL_PROFILE
> >> >   Device Topology (NV)                            PCI-E, 01:00.0
> >> >   Max compute units                               5
> >> >   Max clock frequency                             1150MHz
> >> >   Compute Capability (NV)                         5.0
> >> >   Device Partition                                (core)
> >> >     Max number of sub-devices                     1
> >> >     Supported partition types                     None
> >> >   Max work item dimensions                        3
> >> >   Max work item sizes                             1024x1024x64
> >> >   Max work group size                             1024
> >> >   Preferred work group size multiple              32
> >> >   Warp size (NV)                                  32
> >> >   Preferred / native vector sizes
> >> >     char                                                 1 / 1
> >> >     short                                                1 / 1
> >> >     int                                                  1 / 1
> >> >     long                                                 1 / 1
> >> >     half                                                 0 / 0
> >> (n/a)
> >> >     float                                                1 / 1
> >> >     double                                               1 / 1
> >> >  (cl_khr_fp64)
> >> >   Half-precision Floating-point support           (n/a)
> >> >   Single-precision Floating-point support         (core)
> >> >     Denormals                                     Yes
> >> >     Infinity and NANs                             Yes
> >> >     Round to nearest                              Yes
> >> >     Round to zero                                 Yes
> >> >     Round to infinity                             Yes
> >> >     IEEE754-2008 fused multiply-add               Yes
> >> >     Support is emulated in software               No
> >> >     Correctly-rounded divide and sqrt operations  Yes
> >> >   Double-precision Floating-point support         (cl_khr_fp64)
> >> >     Denormals                                     Yes
> >> >     Infinity and NANs                             Yes
> >> >     Round to nearest                              Yes
> >> >     Round to zero                                 Yes
> >> >     Round to infinity                             Yes
> >> >     IEEE754-2008 fused multiply-add               Yes
> >> >     Support is emulated in software               No
> >> >     Correctly-rounded divide and sqrt operations  No
> >> >   Address bits                                    64, Little-Endian
> >> >   Global memory size                              2095841280
> (1.952GiB)
> >> >   Error Correction support                        No
> >> >   Max memory allocation                           523960320 (499.7MiB)
> >> >   Unified memory for Host and Device              No
> >> >   Integrated memory (NV)                          No
> >> >   Minimum alignment for any data type             128 bytes
> >> >   Alignment of base address                       4096 bits (512
> bytes)
> >> >   Global Memory cache type                        Read/Write
> >> >   Global Memory cache size                        81920
> >> >   Global Memory cache line                        128 bytes
> >> >   Image support                                   Yes
> >> >     Max number of samplers per kernel             32
> >> >     Max size for 1D images from buffer            134217728 pixels
> >> >     Max 1D or 2D image array size                 2048 images
> >> >     Max 2D image size                             16384x16384 pixels
> >> >     Max 3D image size                             4096x4096x4096
> pixels
> >> >     Max number of read image args                 256
> >> >     Max number of write image args                16
> >> >   Local memory type                               Local
> >> >   Local memory size                               49152 (48KiB)
> >> >   Registers per block (NV)                        65536
> >> >   Max constant buffer size                        65536 (64KiB)
> >> >   Max number of constant args                     9
> >> >   Max size of kernel argument                     4352 (4.25KiB)
> >> >   Queue properties
> >> >     Out-of-order execution                        Yes
> >> >     Profiling                                     Yes
> >> >   Prefer user sync for interop                    No
> >> >   Profiling timer resolution                      1000ns
> >> >   Execution capabilities
> >> >     Run OpenCL kernels                            Yes
> >> >     Run native kernels                            No
> >> >     Kernel execution timeout (NV)                 Yes
> >> >   Concurrent copy and kernel execution (NV)       Yes
> >> >     Number of async copy engines                  1
> >> >   printf() buffer size                            1048576 (1024KiB)
> >> >   Built-in kernels
> >> >   Device Available                                Yes
> >> >   Compiler Available                              Yes
> >> >   Linker Available                                Yes
> >> >   Device Extensions
> >> > cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics
> >> > cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics
> >> > cl_khr_fp64 cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing
> >> > cl_nv_compiler_options cl_nv_device_attribute_query
> cl_nv_pragma_unroll
> >> > cl_nv_copy_opts
> >> >
> >> > NULL platform behavior
> >> >   clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...)  NVIDIA CUDA
> >> >   clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...)   Success [NV]
> >> >   clCreateContext(NULL, ...) [default]            Success [NV]
> >> >   clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU)  No devices found
> in
> >> > platform
> >> >   clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU)  No platform
> >> >   clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR)  No
> devices
> >> > found in platform
> >> >   clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM)  No devices
> >> found in
> >> > platform
> >> >   clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL)  No platform
> >> >
> >> > ICD loader properties
> >> >   ICD loader Name                                 OpenCL ICD Loader
> >> >   ICD loader Vendor                               OCL Icd free
> software
> >> >   ICD loader Version                              2.2.8
> >> >   ICD loader Profile                              OpenCL 1.2
> >> > NOTE: your OpenCL library declares to support OpenCL 1.2,
> >> > but it seems to support up to OpenCL 2.1 too.
> >> >
> >>
> >
> >
>

Reply via email to