Re: WARN util.NativeCodeLoader

[Steve Loughran]

> On 8 Dec 2016, at 06:38, baipeng  wrote:
> 
> Hi ALL
> 
> I’m new to Spark.When I execute   spark-shell, the first line is as 
> follows
> WARN util.NativeCodeLoader: Unable to load native-hadoop library for your 
> platform... using builtin-java classes where applicable.
> Can someone tell me how to solve the problem?
> 


It's low level hadoop code warning that it couldn't load the hadoop native 
libraries that are needed for performance, and, on windows, basic 
functionality. 

In the servers, it's a sign that performance working with compressed files is 
going to be worse than it could be, HDFS encryption, and other things.

in your client: it's a distraction.

The fix: turn down logging there

log4j.logger.org.apache.hadoop.util.NativeCodeLoader=ERROR



> -
> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
> 
> 

Re: WARN util.NativeCodeLoader

[Sean Owen]

You can ignore it. You can also install the native libs in question but
it's just a minor accelerator.

On Thu, Dec 8, 2016 at 2:36 PM baipeng  wrote:

> Hi ALL
>
> I’m new to Spark.When I execute spark-shell, the first line is as follows
>  WARN util.NativeCodeLoader: Unable to load native-hadoop library for your
> platform... using builtin-java classes where applicable.
> Can someone tell me how to solve the problem?
>
> -
> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
>
>