The scope should not be provided. Thanks - Gaurav
> On Aug 24, 2015, at 2:09 PM, York, Brennon <[email protected]> > wrote: > > All, seeing some odd behavior when submitting a job through `dtcli` that, I > believe has something to do with Malhar(??). We’re using both > `malhar-contrib` and `malhar-library` in the application and, when both are > provided as `scope: provided`, the app will fail with the below (full stack > at bottom): > > java.lang.NoClassDefFoundError: > com/datatorrent/lib/io/fs/AbstractFileInputOperator > > Now, when we package both into the resultant jar, the application then fails > at the YARN level with a statement like: > > Application application_1440084097966_0009 failed 2 times due to AM Container > for appattempt_1440084097966_0009_000002 exited with exitCode: 1 > > From the YARN Resource Manager. Digging deeper into the YARN container logs > only provides the following exception: > > Exception in thread "main" java.lang.AbstractMethodError: > org.apache.hadoop.yarn.api.records.ContainerId.setContainerId(J)V > at > org.apache.hadoop.yarn.api.records.ContainerId.newInstance(ContainerId.java:60) > at > org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:178) > at com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:87) > > Any thoughts on this and why, when we jar the malhar packages into our app > jar, it fails like this? Could this be a version mismatch? > > For reference we’re using the DT Community Edition and building against > Malhar v3.0.0 as well as DT v3.0.0. > > Full stack trace when `scope: provided` for the Malhar packages: > java.lang.NoClassDefFoundError: > com/datatorrent/lib/io/fs/AbstractFileInputOperator > at java.lang.ClassLoader.defineClass1(Native Method) > at java.lang.ClassLoader.defineClass(ClassLoader.java:800) > at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) > at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) > at java.net.URLClassLoader.access$100(URLClassLoader.java:71) > at java.net.URLClassLoader$1.run(URLClassLoader.java:361) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:793) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > at > com.capitalone.vault8.citadel.H20Application.populateDAG(H20Application.java:40) > at > com.datatorrent.stram.plan.logical.LogicalPlanConfiguration.prepareDAG(LogicalPlanConfiguration.java:1171) > at > com.datatorrent.stram.client.StramAppLauncher$1.createApp(StramAppLauncher.java:404) > at > com.datatorrent.stram.client.StramAppLauncher.launchApp(StramAppLauncher.java:479) > at com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:2048) > at com.datatorrent.stram.cli.DTCli.launchAppPackage(DTCli.java:3452) > at com.datatorrent.stram.cli.DTCli.access$7000(DTCli.java:104) > at com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:1893) > at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1447) > Caused by: java.lang.ClassNotFoundException: > com.datatorrent.lib.io.fs.AbstractFileInputOperator > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:793) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > ... 21 more > ________________________________________________________ > > The information contained in this e-mail is confidential and/or proprietary > to Capital One and/or its affiliates and may only be used solely in > performance of work or services for Capital One. The information transmitted > herewith is intended only for use by the individual or entity to which it is > addressed. If the reader of this message is not the intended recipient, you > are hereby notified that any review, retransmission, dissemination, > distribution, copying or other use of, or taking of any action in reliance > upon this information is strictly prohibited. If you have received this > communication in error, please contact the sender and delete the material > from your computer.
