[ https://issues.apache.org/jira/browse/SPARK-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15056630#comment-15056630 ]
RJ Nowling commented on SPARK-4816: ----------------------------------- Tried with Maven 3.3.9. I see no issues with the newer version of Maven: {code} $ mvn -version Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T16:41:47+00:00) Maven home: /root/apache-maven-3.3.9 Java version: 1.7.0_85, vendor: Oracle Corporation Java home: /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.85-2.6.1.2.el7_1.x86_64/jre Default locale: en_US, platform encoding: UTF-8 OS name: "linux", version: "3.10.0-229.1.2.el7.x86_64", arch: "amd64", family: "unix" $ zipinfo -1 assembly/target/scala-2.10/spark-assembly-1.4.1-hadoop2.4.0.jar | grep netlib-native netlib-native_ref-osx-x86_64.jnilib netlib-native_ref-osx-x86_64.jnilib.asc netlib-native_ref-osx-x86_64.pom netlib-native_ref-osx-x86_64.pom.asc META-INF/maven/com.github.fommil.netlib/netlib-native_ref-osx-x86_64/ META-INF/maven/com.github.fommil.netlib/netlib-native_ref-osx-x86_64/pom.xml META-INF/maven/com.github.fommil.netlib/netlib-native_ref-osx-x86_64/pom.properties netlib-native_ref-linux-x86_64.so META-INF/maven/com.github.fommil.netlib/netlib-native_ref-linux-x86_64/ META-INF/maven/com.github.fommil.netlib/netlib-native_ref-linux-x86_64/pom.xml META-INF/maven/com.github.fommil.netlib/netlib-native_ref-linux-x86_64/pom.properties netlib-native_ref-linux-i686.so META-INF/maven/com.github.fommil.netlib/netlib-native_ref-linux-i686/ META-INF/maven/com.github.fommil.netlib/netlib-native_ref-linux-i686/pom.xml META-INF/maven/com.github.fommil.netlib/netlib-native_ref-linux-i686/pom.properties netlib-native_ref-win-x86_64.dll netlib-native_ref-win-x86_64.dll.asc netlib-native_ref-win-x86_64.pom netlib-native_ref-win-x86_64.pom.asc META-INF/maven/com.github.fommil.netlib/netlib-native_ref-win-x86_64/ META-INF/maven/com.github.fommil.netlib/netlib-native_ref-win-x86_64/pom.xml META-INF/maven/com.github.fommil.netlib/netlib-native_ref-win-x86_64/pom.properties netlib-native_ref-win-i686.dll netlib-native_ref-win-i686.dll.asc netlib-native_ref-win-i686.pom netlib-native_ref-win-i686.pom.asc META-INF/maven/com.github.fommil.netlib/netlib-native_ref-win-i686/ META-INF/maven/com.github.fommil.netlib/netlib-native_ref-win-i686/pom.xml META-INF/maven/com.github.fommil.netlib/netlib-native_ref-win-i686/pom.properties netlib-native_ref-linux-armhf.so META-INF/maven/com.github.fommil.netlib/netlib-native_ref-linux-armhf/ META-INF/maven/com.github.fommil.netlib/netlib-native_ref-linux-armhf/pom.xml META-INF/maven/com.github.fommil.netlib/netlib-native_ref-linux-armhf/pom.properties netlib-native_system-osx-x86_64.jnilib netlib-native_system-osx-x86_64.jnilib.asc netlib-native_system-osx-x86_64.pom netlib-native_system-osx-x86_64.pom.asc META-INF/maven/com.github.fommil.netlib/netlib-native_system-osx-x86_64/ META-INF/maven/com.github.fommil.netlib/netlib-native_system-osx-x86_64/pom.xml META-INF/maven/com.github.fommil.netlib/netlib-native_system-osx-x86_64/pom.properties netlib-native_system-linux-x86_64.pom.asc netlib-native_system-linux-x86_64.pom netlib-native_system-linux-x86_64.so netlib-native_system-linux-x86_64.so.asc META-INF/maven/com.github.fommil.netlib/netlib-native_system-linux-x86_64/ META-INF/maven/com.github.fommil.netlib/netlib-native_system-linux-x86_64/pom.xml META-INF/maven/com.github.fommil.netlib/netlib-native_system-linux-x86_64/pom.properties netlib-native_system-linux-i686.pom netlib-native_system-linux-i686.so.asc netlib-native_system-linux-i686.pom.asc netlib-native_system-linux-i686.so META-INF/maven/com.github.fommil.netlib/netlib-native_system-linux-i686/ META-INF/maven/com.github.fommil.netlib/netlib-native_system-linux-i686/pom.xml META-INF/maven/com.github.fommil.netlib/netlib-native_system-linux-i686/pom.properties netlib-native_system-linux-armhf.pom netlib-native_system-linux-armhf.so.asc netlib-native_system-linux-armhf.pom.asc netlib-native_system-linux-armhf.so META-INF/maven/com.github.fommil.netlib/netlib-native_system-linux-armhf/ META-INF/maven/com.github.fommil.netlib/netlib-native_system-linux-armhf/pom.xml META-INF/maven/com.github.fommil.netlib/netlib-native_system-linux-armhf/pom.properties netlib-native_system-win-x86_64.dll netlib-native_system-win-x86_64.dll.asc netlib-native_system-win-x86_64.pom netlib-native_system-win-x86_64.pom.asc META-INF/maven/com.github.fommil.netlib/netlib-native_system-win-x86_64/ META-INF/maven/com.github.fommil.netlib/netlib-native_system-win-x86_64/pom.xml META-INF/maven/com.github.fommil.netlib/netlib-native_system-win-x86_64/pom.properties netlib-native_system-win-i686.dll netlib-native_system-win-i686.dll.asc netlib-native_system-win-i686.pom netlib-native_system-win-i686.pom.asc META-INF/maven/com.github.fommil.netlib/netlib-native_system-win-i686/ META-INF/maven/com.github.fommil.netlib/netlib-native_system-win-i686/pom.xml META-INF/maven/com.github.fommil.netlib/netlib-native_system-win-i686/pom.properties {code} Thus, it's likely only a problem with older versions of Maven or its plugins. Maven 3.0+ is support for Spark 1.4.1 according to the docs. Even without the commit that fixed the issue, this probably wouldn't be an issue with 1.5.x+ since they only support Maven 3.3.x+. > Maven profile netlib-lgpl does not work > --------------------------------------- > > Key: SPARK-4816 > URL: https://issues.apache.org/jira/browse/SPARK-4816 > Project: Spark > Issue Type: Bug > Components: Build > Affects Versions: 1.1.0 > Environment: maven 3.0.5 / Ubuntu > Reporter: Guillaume Pitel > Priority: Minor > Fix For: 1.1.1 > > > When doing what the documentation recommends to recompile Spark with Netlib > Native system binding (i.e. to bind with openblas or, in my case, MKL), > mvn -Pnetlib-lgpl -Pyarn -Phadoop-2.3 -Dhadoop.version=2.3.0 -DskipTests > clean package > The resulting assembly jar still lacked the netlib-system class. (I checked > the content of spark-assembly...jar) > When forcing the netlib-lgpl profile in MLLib package to be active, the jar > is correctly built. > So I guess it's a problem with the way maven passes profiles activitations to > children modules. > Also, despite the documentation claiming that if the job's jar contains > netlib with necessary bindings, it should works, it does not. The classloader > must be unhappy with two occurrences of netlib ? -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org