Hi Aakash, Sorry for the slow response. Do you have any additional logs that show the exception? If possible, could you reproduce this and provide the code so I can try it out?
Alvin On Tue, Apr 8, 2014 at 5:17 PM, Terence Yim <[email protected]> wrote: > Hi Aakash, > > Thanks for the finding. I've created a JIRA for investigating the problem. > > https://issues.apache.org/jira/browse/TWILL-65 > > Terence > > On Sun, Apr 6, 2014 at 5:00 PM, aakash aakash <[email protected]> > wrote: > > Thanks Alvin!! > > > > I tried the solution, you suggested but application is throwing > exception : > > > > 14/04/06 14:57:10 INFO internal.ServiceMain: Service > > org.apache.twill.internal.appmaster.ApplicationMasterService@3c9c31 > > started. > > 14/04/06 14:57:10 INFO controller.ControllerEpochListener: > > [ControllerEpochListener on 1]: Initialized controller epoch to 1 and > > zk version 0 > > 14/04/06 14:57:10 INFO appmaster.ApplicationMasterService: Request 1 > > container with capability <memory:512, vCores:1> > > 14/04/06 14:57:12 INFO appmaster.ApplicationMasterService: Got > > container container_1396815710754_0007_01_000002 > > 14/04/06 14:57:12 INFO appmaster.ApplicationMasterService: Starting > > runnable BundledJarRunnable with > > > RunnableProcessLauncher{container=org.apache.twill.internal.yarn.Hadoop21YarnContainerInfo@1f30035 > } > > 14/04/06 14:57:12 INFO appmaster.RunnableProcessLauncher: Launching in > > container container_1396815710754_0007_01_000002 at > > localhost/127.0.0.1, [java -Djava.io.tmpdir=tmp > > -Dyarn.container=$YARN_CONTAINER_ID > > -Dtwill.runnable=$TWILL_APP_NAME.$TWILL_RUNNABLE_NAME -cp > > launcher.jar:$HADOOP_CONF_DIR -Xmx359m > > org.apache.twill.launcher.TwillLauncher container.jar > > org.apache.twill.internal.container.TwillContainerMain true > > 1><LOG_DIR>/stdout 2><LOG_DIR>/stderr] > > 14/04/06 14:57:12 INFO impl.ContainerManagementProtocolProxy: Opening > > proxy : localhost:55102 > > 14/04/06 14:57:12 INFO appmaster.ApplicationMasterService: Runnable > > BundledJarRunnable fully provisioned with 1 instances. > > 14/04/06 14:57:18 INFO appmaster.ApplicationMasterService: Container > > container_1396815710754_0007_01_000002 completed with > > COMPLETE:Exception from container-launch: > > org.apache.hadoop.util.Shell$ExitCodeException: > > at org.apache.hadoop.util.Shell.runCommand(Shell.java:464) > > at org.apache.hadoop.util.Shell.run(Shell.java:379) > > at > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589) > > at > org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195) > > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283) > > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79) > > at > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) > > at java.lang.Thread.run(Thread.java:662) > > > > > > . > > 14/04/06 14:57:18 WARN appmaster.RunningContainers: Container > > container_1396815710754_0007_01_000002 exited abnormally with state > > COMPLETE, exit code 10. > > 14/04/06 14:57:18 INFO appmaster.ApplicationMasterService: All > > containers completed. Shutting down application master. > > 14/04/06 14:57:18 INFO appmaster.ApplicationMasterService: Stop > > application master with spec: > > > {"name":"ExampleBundledJarApp","runnables":{"BundledJarRunnable":{"name":"BundledJarRunnable","runnable":{"classname":"org.apache.twill.ext.BundledJarRunnable","name":"BundledJarRunnable","arguments":{}},"resources":{"cores":1,"memorySize":512,"instances":1,"uplink":-1,"downlink":-1,"hosts":[],"racks":[]},"files":[{"name":"job-web-service-1.0.jar","uri":"file:/twill/ExampleBundledJarApp/03f7c49a-6471-40b6-9992-af07905d44cc/job-web-service-1.0.jar.7e6aaa7d-fc66-462d-abeb-3b2fb3b7d812.jar","lastModified":1396821425000,"size":75347840,"archive":false,"pattern":null}]}},"orders":[{"names":["BundledJarRunnable"],"type":"STARTED"}],"handler":{"classname":"org.apache.twill.internal.LogOnlyEventHandler","configs":{}}} > > 14/04/06 14:57:18 INFO appmaster.RunningContainers: Stopping all > > instances of BundledJarRunnable > > > > > > > > > > > > Twill example code: > > > > ------------------- > > > > > > > > //Example Application > > > > > > private static class ExampleBundledJarApp implements TwillApplication { > > 77 private final String jarName; > > 78 private final URI jarURI; > > 79 > > 80 public ExampleBundledJarApp(String jarName, URI > jarURI) { > > 81 this.jarName = jarName; > > 82 this.jarURI = jarURI; > > 83 } > > 84 > > 85 @Override > > 86 public TwillSpecification configure() { > > 87 return TwillSpecification.Builder.with() > > 88 > > .setName("ExampleBundedJarApp").withRunnable() > > 89 .add("BundledJarRunnable", > > new BundledJarRunnable()) > > 90 > > .withLocalFiles().add(jarName, jarURI, false).apply() > > 91 .anyOrder().build(); > > 92 } > > 93 } > > > > > > > > > > > > //calling twill runner > > > > Arguments arguments = new Arguments.Builder() > > 204 > > .setJarFileName("lib/job-web-service-1.0.jar").setLibFolder("lib") > > 205 > > .setMainClassName(Main.class.getPackage().getName()).setMainArgs(new > > String[]{"1"}) > > 206 .createArguments(); > > 207 > > 208 > > 209 > > 210 LOG.info("arguments jar file:: "+ > > arguments.getJarFileName()); > > 211 LOG.info("arguments lib folder:: "+ > > arguments.getLibFolder()); > > 212 LOG.info("arguments main class:: "+ > > arguments.getMainClassName()); > > 213 LOG.info("arguments main args:: "+ > > arguments.getMainArgs()); > > 214 > > 215 LOG.info("arguments:: "+ > > Arrays.asList(arguments.toArray())); > > 216 > > 217 > > 218 File jarFile = new File(arguments.getJarFileName()); > > 219 Preconditions.checkArgument(jarFile != null); > > 220 Preconditions.checkState(jarFile.exists(), "jar file > > does not exist: "+jarFile.getAbsolutePath()); > > 221 Preconditions.checkState(jarFile.canRead()); > > 222 > > 223 > > 224 TwillPreparer twillPreparer = runnerService > > 225 .prepare(new > > ExampleBundledJarApp(jarFile.getName(), jarFile.toURI())) > > 226 .withArguments("BundledJarRunnable", > > arguments.toArray()) > > 227 .addLogHandler( > > 228 new > > PrinterLogHandler(new PrintWriter(System.out))); > > 229 > > 230 twillPreparer.start(); > > > > > > There are two problem I observed here: > > > > 1. if u dont set any main arg it throw an exception, need to be fixed by > > BundledJarRunnable.Arguments by checking main args for being null > > 2. I see arguments are not populated in the json (check the container > error > > log above). > > > > > > Thanks, > > Aakash > > > > > > > > > > > > > > On Tue, Apr 1, 2014 at 4:35 PM, Alvin Wang <[email protected]> > wrote: > > > >> I recently committed something that can serve as a workaround for this, > >> when I was trying to get Presto running with Twill. Presto was using > Guava > >> 16 which was incompatible with Twill's Guava 13. > >> > >> You can take a look at BundledJarRunnable available in the > 0.2.0-incubating > >> release (sample > >> usage< > >> > https://git-wip-us.apache.org/repos/asf?p=incubator-twill.git;a=blob;f=twill-examples/yarn/src/main/java/org/apache/twill/example/yarn/BundledJarExample.java;h=3cc88a1b7624270c2d870c8a249185d608b09613;hb=d4a352de4a62abdf4607af77d610597630e0ee1a > >> >). > >> It's currently very limited in terms of functionality, but you can use > it > >> to run an application without having to worry about Twill's classpath. > You > >> can specify a bundle jar file, folder containing the dependencies within > >> the jar file, main class name, and arguments to send to the main method. > >> > >> The BundledJarRunnable will load the jars in the folder containing the > >> dependencies and also the bundle jar into a clean classloader, so it > won't > >> conflict with Twill's dependencies. To use this, do the following: > >> > >> 1. Package your program into a bundle jar. I used the following in my > >> pom.xml: > >> > >> <build> > >> <plugins> > >> <plugin> > >> <groupId>org.apache.felix</groupId> > >> <artifactId>maven-bundle-plugin</artifactId> > >> <version>2.3.7</version> > >> <extensions>true</extensions> > >> <configuration> > >> <instructions> > >> > <Embed-Dependency>*;inline=false</Embed-Dependency> > >> <Embed-Transitive>true</Embed-Transitive> > >> <Embed-Directory>lib</Embed-Directory> > >> </instructions> > >> </configuration> > >> </plugin> > >> </plugins> > >> </build> > >> > >> 2. Configure your TwillApplication to include a BundledJarRunnable() and > >> the jar file: > >> ...withRunnable().add("Test123", new > >> BundledJarRunnable()).withLocalFiles().add("sdf.jar", localJarURI, > false) > >> > >> 3. Configure your TwillPreparer to pass runtime arguments (main class > name, > >> main method args, etc.) to your runnable: > >> > >> BundledJarRunner.Arguments arguments = new > >> BundledJarRunner.Arguments.Builder() > >> .set ... > >> ... > >> .build(); > >> > >> final TwillController controller = twillRunner.prepare( > >> new ExampleBundledJarApp() > >> .withArguments("Test123", arguments.toArray()) > >> .addLogHandler(new PrinterLogHandler(new PrintWriter(System.out, > >> true))) > >> .start(); > >> > >> -- > >> > >> Let me know if you have any questions. > >> > >> Alvin > >> > >> > >> On Tue, Apr 1, 2014 at 3:01 AM, aakash aakash <[email protected] > >> >wrote: > >> > >> > Hi, > >> > > >> > I am trying to start a rest web service on hadoop using Twill, but > >> > unfortunately jersey use asm 3.1 but in Twill we use asm 4 and > source is > >> > also different. Which is now a big problem for me. > >> > > >> > Is there any workaround for this. Any help will be highly appreciated. > >> > > >> > Thanks, > >> > Aakash > >> > > >> >
