[ https://issues.apache.org/jira/browse/AMBARI-25987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robin Zuo reassigned AMBARI-25987: ---------------------------------- Assignee: Robin Zuo > Enable Spark to use hadoop native libraries for better performance > ------------------------------------------------------------------ > > Key: AMBARI-25987 > URL: https://issues.apache.org/jira/browse/AMBARI-25987 > Project: Ambari > Issue Type: Improvement > Components: ambari-server > Reporter: Robin Zuo > Assignee: Robin Zuo > Priority: Major > > Currently, when run spark tasks, spark will report cannot load hadoop native > libraries. > Also, the hadoop short-circuit read feature cannot be applied. For better > performance, add hadoop native libraries support. > Some WARN log as following: > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] > 23/08/15 13:43:22 WARN NativeCodeLoader: Unable to load native-hadoop library > for your platform... using builtin-java classes where applicable > ... > 23/08/15 13:43:25 INFO Client: Requesting a new application from cluster with > 3 NodeManagers > 23/08/15 13:43:26 WARN DomainSocketFactory: The short-circuit local reads > feature cannot be used because libhadoop cannot be loaded. > 23/08/15 13:43:26 INFO Configuration: resource-types.xml not found > -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@ambari.apache.org For additional commands, e-mail: issues-h...@ambari.apache.org