[ https://issues.apache.org/jira/browse/MAHOUT-1934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15850887#comment-15850887 ]
ASF GitHub Bot commented on MAHOUT-1934: ---------------------------------------- GitHub user andrewpalumbo opened a pull request: https://github.com/apache/mahout/pull/273 MAHOUT-1934: OpenMP jars aren't being picked up add in dependency-reduced.jar to put javacpp jars on executor classpath. Hack for now. use simple wildcard as well. You can merge this pull request into a Git repository by running: $ git pull https://github.com/andrewpalumbo/mahout MAHOUT-1934b Alternatively you can review and apply these changes as the patch at: https://github.com/apache/mahout/pull/273.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #273 ---- commit 892f570b879ccdedd59ad03eb6690da661b90f2c Author: Andrew Palumbo <apalu...@apache.org> Date: 2017-02-03T01:29:15Z MAHOUT-1934: OpenMP jars aren't being picked up add in dependency-reduced.jar to put javacpp jars on executor classpath. Hack for now. ---- > OpenMP jars aren't being picked up in distributed Spark > ------------------------------------------------------- > > Key: MAHOUT-1934 > URL: https://issues.apache.org/jira/browse/MAHOUT-1934 > Project: Mahout > Issue Type: Improvement > Affects Versions: 0.13.0 > Reporter: Trevor Grant > Assignee: Andrew Palumbo > Priority: Blocker > Fix For: 0.13.0 > > > When executing the following: > ``` > import org.apache.mahout.math.decompositions._ > drmX.checkpoint() > val (U, V, s) = dspca(drmX, k = 30, q = 1) > ``` > I see the following: > ``` > import org.apache.mahout.math.decompositions._ > res1: org.apache.mahout.math.drm.CheckpointedDrm[Int] = > org.apache.mahout.sparkbindings.drm.CheckpointedDrmSpark@2e8dd198 > [INFO] Creating org.apache.mahout.viennacl.opencl.GPUMMul solver > [WARN] Unable to create class GPUMMul: attempting OpenMP version > [INFO] Creating org.apache.mahout.viennacl.openmp.OMPMMul solver > org.apache.mahout.viennacl.openmp.OMPMMul$ > [INFO] Unable to create class OMPMMul: falling back to java version > drmReduxX: org.apache.mahout.math.drm.DrmLike[Int] = > OpMapBlock(OpTimesRightMatrix(org.apache.mahout.s... > ``` > The code executes fine, but it seems the OpenMP is not getting picked up. > To be clear I build these jars with -Pviennacl-omp > I the issue persists through other operations as well. I am running on a 3 > node Yarn cluster/ spark 1.6.3 -- This message was sent by Atlassian JIRA (v6.3.15#6346)