[ https://issues.apache.org/jira/browse/AMBARI-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15431844#comment-15431844 ]
Hudson commented on AMBARI-18112: --------------------------------- FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #5572 (See [https://builds.apache.org/job/Ambari-trunk-Commit/5572/]) AMBARI-18112. Fix spark.executor.extraLibraryPath to include native gpl (smohanty: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=3b600322937f3a0c95f3079c08bd3a9580f535c4]) * (edit) ambari-server/src/main/resources/common-services/SPARK2/2.0.0/configuration/spark2-defaults.xml * (edit) ambari-server/src/main/resources/common-services/SPARK/1.2.1/configuration/spark-defaults.xml * (edit) ambari-server/src/main/resources/common-services/SPARK/1.5.2/configuration/spark-thrift-sparkconf.xml * (edit) ambari-server/src/main/resources/common-services/SPARK2/2.0.0/configuration/spark2-thrift-sparkconf.xml > Fix spark.executor.extraLibraryPath to include native gpl library > ----------------------------------------------------------------- > > Key: AMBARI-18112 > URL: https://issues.apache.org/jira/browse/AMBARI-18112 > Project: Ambari > Issue Type: Bug > Components: ambari-server > Affects Versions: trunk, 2.4.0, 2.5.0 > Reporter: Weiqing Yang > Assignee: Weiqing Yang > Fix For: trunk > > Attachments: AMBARI-18112_v0.patch, AMBARI-18112_v1.patch > > > When spark applications run in yarn cluster mode, there is an error from > executors: > --------------------------------------------------------------------------------- > java.lang.UnsatisfiedLinkError: no gplcompression in java.library.path > at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1889) > at java.lang.Runtime.loadLibrary0(Runtime.java:849) > at java.lang.System.loadLibrary(System.java:1088) > at > com.hadoop.compression.lzo.GPLNativeCodeLoader.<clinit>(GPLNativeCodeLoader.java:32) > at com.hadoop.compression.lzo.LzoCodec.<clinit>(LzoCodec.java:71) > at java.lang.Class.forName0(Native Method) > ....... > org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > 16/08/08 12:59:03 ERROR LzoCodec: Cannot load native-lzo without > native-hadoop. > In the patch, it will set “spark.executor.extraLibraryPath > /usr/hdp/current/hadoop-client/lib/native:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64” > in spark-defaults.conf. -- This message was sent by Atlassian JIRA (v6.3.4#6332)