[
https://issues.apache.org/jira/browse/GEARPUMP-147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15293123#comment-15293123
]
Karol Brejna commented on GEARPUMP-147:
---------------------------------------
[~HuafengWang] Adding the DAG jar to worker's classpath (by phisically placing
it there) did the trick. Now the application is running fine!
If the behavior is intentional (it kind of make sense), we should document it,
I've opened ticket for this
(https://issues.apache.org/jira/browse/GEARPUMP-143). Are there any other
assumptions, consequences? (I assume, for example, that
gearpump.executor.vmargs is then ignored...)
I could take the info together and put it
http://gearpump.apache.org/releases/latest/deployment-configuration.html.
> java.lang.ClassNotFoundException when running on worker with
> worker.executor-share-same-jvm-as-worker = true
> ------------------------------------------------------------------------------------------------------------
>
> Key: GEARPUMP-147
> URL: https://issues.apache.org/jira/browse/GEARPUMP-147
> Project: Apache Gearpump
> Issue Type: Task
> Reporter: Karol Brejna
> Assignee: Huafeng Wang
> Attachments: gear.pc.conf, gear.pc.slimworker.conf,
> logs.slimworker.zip, prepare.sh
>
>
> Doing GEARPUMP-146 I hit a problem: I was able to run a master on pc and a
> worker on raspi. After deploying wordcountjava-2.11-0.8.0-assembly.jar the
> example didn't start.
> I reproduced this on "pure pc" cluster.
> I've prepared dedicated config files:
> * gear.pc.conf - normal (default) configuration for
> master and workers ran on a PC
> * gear.pc.slimworker.conf - intends to have lower requirements for memory,
> cpu - to be used on a device (raspi, edison)
> *Obtaining binary and creating config files*
> {code}
> wget
> https://github.com/gearpump/gearpump/releases/download/0.8.0/gearpump-2.11-0.8.0.zip
> unzip gearpump-2.11-0.8.0.zip
> cd gearpump-2.11-0.8.0
> cp conf/gear.conf conf/gear.original.conf
> cp conf/gear.conf conf/gear.pc.conf
> cp conf/gear.conf conf/gear.pc.slimworker.conf
> {code}
> *Preparing "PC" conf*
> The only real change here is putting the IP of the machine that will run the
> master:
> {code}
> cluster {
> masters = ["192.168.1.115:3000"]
> }
>
> hostname = "192.168.1.115"
> {code}
>
> *Preparing device conf*
> Compared to previous config, this config should set up less worker slots,
> smaller -Xmx jvm switch for worker, and executor-share-same-jvm-as-worker set
> to true:
> {code}
> worker.slots = 100
> worker.executor-share-same-jvm-as-worker = true
> executor {
> vmargs = "-server -Xms512M -Xmx512M -Xss1M
> -XX:+HeapDumpOnOutOfMemoryError -XX:+UseConcMarkSweepGC
> -XX:CMSInitiatingOccupancyFraction=80 -XX:+UseParNewGC -XX:NewRatio=3
> -Djava.rmi.server.hostname=localhost"
> extraClasspath = ""
> }
> {code}
> *Running the cluster:*
> {code}
> echo "Starting the master"
> cp conf/gear.pc.conf conf/gear.conf
> bin/master -ip 192.168.1.115 -port 3000 &
> echo "Starting services"
> bin/services &
> echo "Starting a worker (configured as on the device)"
> cp conf/gear.pc.slimworker.conf conf/gear.conf
> bin/worker &
> {code}
> *Deploing the DAG*
> I've deployed wordcountjava-2.11-0.8.0-assembly.jar.
> The app deploys, but doesn't run.
> (After changing worker.executor-share-same-jvm-as-worker to false and
> restarting the cluster, example dag works.)
> See the attached logs:
> {code}
> 2016-05-18 08:47:31,453 ERROR OneForOneStrategy:
> io.gearpump.streaming.examples.wordcountjava.Split
> java.lang.ClassNotFoundException:
> io.gearpump.streaming.examples.wordcountjava.Split
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at io.gearpump.streaming.task.TaskUtil$.loadClass(TaskUtil.scala:30)
> at
> io.gearpump.streaming.executor.TaskLauncher.launch(TaskLauncher.scala:55)
> at
> io.gearpump.streaming.executor.Executor$$anonfun$dynamicDagPhase1$1.applyOrElse(Executor.scala:147)
> at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170)
> at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
> at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
> at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
> at akka.actor.Actor$class.aroundReceive(Actor.scala:480)
> at
> io.gearpump.streaming.executor.Executor.aroundReceive(Executor.scala:55)
> at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
> at akka.actor.ActorCell.invoke(ActorCell.scala:495)
> at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
> at akka.dispatch.Mailbox.run(Mailbox.scala:224)
> at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
> at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> at
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> at
> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> at
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> {code}
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
