Such error will be raised when hadoop-common and hadoop-hdfs libraries
on the classpath come from different hadoop version. Either cluster is
misconfigured or one of those files is included into the application
package.
Thank you,
Vlad
/Join us at Apex Big Data World-San Jose
<http://www.apexbigdata.com/san-jose.html>, April 4, 2017/
http://www.apexbigdata.com/san-jose-register.html
<http://www.apexbigdata.com/san-jose-register.html>
On 3/3/17 14:18, Sandesh Hegde wrote:
Please check the Hadoop dependency version in your POM. Also we need to
move these discussions to users@
On Fri, Mar 3, 2017 at 2:14 PM Ganelin, Ilya <ilya.gane...@capitalone.com>
wrote:
Minor amendment: hadoop-2.6.0+cdh5.8.0+1592 (2.6 vs 2.7)
- Ilya Ganelin
[image: id:image001.png@01D1F7A4.F3D42980]
*From: *"Ganelin, Ilya" <ilya.gane...@capitalone.com>
*Reply-To: *"dev@apex.apache.org" <dev@apex.apache.org>
*Date: *Friday, March 3, 2017 at 1:59 PM
*To: *"dev@apex.apache.org" <dev@apex.apache.org>
*Subject: *NO_LOCAL_WRITE Error from Stram
Hello, all – new error cropping up at Application start and Google does
not offer any helpful guidance.
2017-03-03 16:49:02,558 INFO
com.datatorrent.common.util.AsyncFSStorageAgent: using
/opt/cloudera/hadoop/1/yarn/nm/usercache/XXXXXX/appcache/application_1483979920683_0448/container_e97905_1483979920683_0448_01_000001/tmp/chkp3008940656673217935
as the basepath for checkpointing.
2017-03-03 16:49:02,739 ERROR com.datatorrent.stram.StreamingAppMaster:
Exiting Application Master
2017-03-03 16:49:02,739 ERROR com.datatorrent.stram.StreamingAppMaster:
Exiting Application Master
java.lang.NoSuchFieldError: NO_LOCAL_WRITE
at
org.apache.hadoop.hdfs.DFSOutputStream.<init>(DFSOutputStream.java:1909)
at
org.apache.hadoop.hdfs.DFSOutputStream.<init>(DFSOutputStream.java:1938)
at
org.apache.hadoop.hdfs.DFSOutputStream.newStreamForCreate(DFSOutputStream.java:1996)
at
org.apache.hadoop.hdfs.DFSClient.primitiveCreate(DFSClient.java:1786)
at org.apache.hadoop.fs.Hdfs.createInternal(Hdfs.java:105)
at org.apache.hadoop.fs.Hdfs.createInternal(Hdfs.java:59)
at
org.apache.hadoop.fs.AbstractFileSystem.create(AbstractFileSystem.java:577)
at
org.apache.hadoop.fs.FileContext$3.next(FileContext.java:680)
at
org.apache.hadoop.fs.FileContext$3.next(FileContext.java:676)
at
org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
at
org.apache.hadoop.fs.FileContext.create(FileContext.java:676)
at
com.datatorrent.common.util.AsyncFSStorageAgent.copyToHDFS(AsyncFSStorageAgent.java:118)
at
com.datatorrent.stram.plan.physical.PhysicalPlan.initCheckpoint(PhysicalPlan.java:1236)
at
com.datatorrent.stram.plan.physical.PhysicalPlan.<init>(PhysicalPlan.java:495)
at
com.datatorrent.stram.StreamingContainerManager.<init>(StreamingContainerManager.java:418)
at
com.datatorrent.stram.StreamingContainerManager.getInstance(StreamingContainerManager.java:3065)
at
com.datatorrent.stram.StreamingAppMasterService.serviceInit(StreamingAppMasterService.java:552)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:102)
Hadoop version is 2.7.2, Apex-core 3.5.0, Gateway 3.7.0, Java 1.8
Any thoughts? Thanks!
- Ilya Ganelin
------------------------------
The information contained in this e-mail is confidential and/or
proprietary to Capital One and/or its affiliates and may only be used
solely in performance of work or services for Capital One. The information
transmitted herewith is intended only for use by the individual or entity
to which it is addressed. If the reader of this message is not the intended
recipient, you are hereby notified that any review, retransmission,
dissemination, distribution, copying or other use of, or taking of any
action in reliance upon this information is strictly prohibited. If you
have received this communication in error, please contact the sender and
delete the material from your computer.
------------------------------
The information contained in this e-mail is confidential and/or
proprietary to Capital One and/or its affiliates and may only be used
solely in performance of work or services for Capital One. The information
transmitted herewith is intended only for use by the individual or entity
to which it is addressed. If the reader of this message is not the intended
recipient, you are hereby notified that any review, retransmission,
dissemination, distribution, copying or other use of, or taking of any
action in reliance upon this information is strictly prohibited. If you
have received this communication in error, please contact the sender and
delete the material from your computer.
