Hi Joe,
Please find the contents of core-site.xml which contains following
property.
<property>
<name>io.compression.codecs</name>
<value>org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.SnappyCodec</value>
</property>
mapred-site.xml
<property>
<name>mapreduce.map.output.compress</name>
<value>true</value>
</property>
<property>
<name>mapred.map.output.compress.codec</name>
<value>org.apache.hadoop.io.compress.SnappyCodec</value>
</property>
<property>
<name>mapreduce.admin.user.env</name>
<value>LD_LIBRARY_PATH=/usr/hdp/2.2.0.0-1084/hadoop/lib/native</value>
</property>
The logs snapshot as following:-
2016-02-09 15:03:14,713 INFO [pool-26-thread-8]
o.apache.nifi.processors.hadoop.PutHDFS
PutHDFS[id=1a396460-dca5-4adf-af4a-8bc828c2e395] Initialized a new HDFS File
System with working dir: hdfs://localhost:9000/user/shweta default block
size: 134217728 default replication: 1 config: Configuration:
core-default.xml, core-site.xml, mapred-default.xml, mapred-site.xml,
yarn-default.xml, yarn-site.xml, hdfs-default.xml, hdfs-site.xml,
/home/shweta/software/hadoop/hadoop-2.4.0/etc/hadoop/hdfs-site.xml,
/home/shweta/software/hadoop/hadoop-2.4.0/etc/hadoop/core-site.xml
2016-02-09 15:03:14,724 INFO [pool-26-thread-8]
o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
PutHDFS[id=1a396460-dca5-4adf-af4a-8bc828c2e395] to run with 1 threads
2016-02-09 15:03:15,999 INFO [Timer-Driven Process Thread-8]
o.a.nifi.processors.standard.ReplaceText
ReplaceText[id=59f04d7a-c230-4774-a78f-90f44bcc24d5] Transferred
StandardFlowFileRecord[uuid=cb84f64f-1bc1-4922-8380-b2692afaf1f5,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1455009982642-1, container=default,
section=1], offset=724356, length=3710],offset=0,name=test2.csv,size=3710]
to 'success'
2016-02-09 15:03:16,015 INFO [Timer-Driven Process Thread-8]
o.a.nifi.processors.standard.ReplaceText
ReplaceText[id=59f04d7a-c230-4774-a78f-90f44bcc24d5] Transferred
StandardFlowFileRecord[uuid=f7156b6b-0a5d-4ea8-9494-0d7bcd2cf83d,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1455009982642-1, container=default,
section=1], offset=728066, length=7],offset=0,name=test.txt,size=7] to
'success'
2016-02-09 15:03:16,282 ERROR [Timer-Driven Process Thread-8]
o.apache.nifi.processors.hadoop.PutHDFS
PutHDFS[id=1a396460-dca5-4adf-af4a-8bc828c2e395] Failed to write to HDFS due
to java.lang.UnsatisfiedLinkError:
org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z
2016-02-09 15:03:17,508 ERROR [Timer-Driven Process Thread-6]
o.apache.nifi.processors.hadoop.PutHDFS
PutHDFS[id=1a396460-dca5-4adf-af4a-8bc828c2e395] Failed to write to HDFS due
to java.lang.UnsatisfiedLinkError:
org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z
Thanks,
Shweta
--
View this message in context:
http://apache-nifi-developer-list.39713.n7.nabble.com/java-lang-UnsatisfiedLinkError-in-PutHDFS-with-snappy-compression-tp7182p7235.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
