Please also check application package (.apa) for any hadoop dependency.
They all must be excluded in the application pom as Apex will provide
them at run-time.
Thank you,
Vlad
On 9/21/16 08:27, Vlad Rozov wrote:
Is gateway/apex installed on edge node of mapr cluster? Most likely
this is compatibility issue between jars installed on gateway and the
actual cluster. The root cause of the application failure is the
following exception on Apex application master
java.lang.NoSuchFieldError: IS_WINDOWS
at
org.apache.hadoop.security.login.GenericOSLoginModule.getOSLoginModuleName(GenericOSLoginModule.java:56)
at
org.apache.hadoop.security.login.GenericOSLoginModule.<clinit>(GenericOSLoginModule.java:64)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at
javax.security.auth.login.LoginContext.invoke(LoginContext.java:710)
at
javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
at java.security.AccessController.doPrivileged(Native Method)
at
javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
at
org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:724)
at
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:688)
at
org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:572)
at
org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:447)
at
com.datatorrent.common.util.FSStorageAgent.<init>(FSStorageAgent.java:84)
at
com.datatorrent.common.util.AsyncFSStorageAgent.<init>(AsyncFSStorageAgent.java:64)
at
com.datatorrent.common.util.AsyncFSStorageAgent.readResolve(AsyncFSStorageAgent.java:151)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at
java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1104)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1810)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
at java.util.HashMap.readObject(HashMap.java:1396)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at
java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1900)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000)
at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000)
at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
at
com.datatorrent.stram.plan.logical.LogicalPlan.read(LogicalPlan.java:2324)
at
com.datatorrent.stram.StreamingAppMasterService.serviceInit(StreamingAppMasterService.java:541)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:102)
Vlad
On 9/20/16 23:23, Bhupesh Chawda wrote:
Hi Jaspal,
I think the issue might be related to using mapr specific jars in the
application package.
Can you check if you are able to run any other app on the cluster;
for ex.pidemo in malhar?
Also check the jars included in your apa file using "mvn
dependency:tree" and verify that no hadoop jars have been bundled.
~ Bhupesh
On Wed, Sep 21, 2016 at 1:45 AM, Jaspal Singh
<jaspal.singh1...@gmail.com <mailto:jaspal.singh1...@gmail.com>> wrote:
Hi Team,
We are trying to build a DT application to read messages from
mapr streams and write them on to hdfs. Its the same application
used with Kafka as source with the following changes:
1. Added the map example code jar in the pom.xml.
2. Changed the topic name to fully qualified path of the stream
with topic name.
After packaging the code into apa file, we are able to launch the
application but it remains in ACCEPTED state and then goes to
FAILED state after a while. We are having a hard time time to
figure out the issue with the project, so need some assistance
from anyone who has encountered any similar issue.
dtgateway.log and pom.xml attached.
Really appreciate the inputs !!
Thanks
Jaspal
