Re: Question for Mesos gurus
Thank you Adam When you say:but before you upgrade Mesos to 0.23, you should upgrade your scheduler (and executor) libmesos to 0.22.x Do you mean - recompile? Does this sentence from link with upgrade instructions you provided means the same? Rebuild and install any modules so that upgraded masters/slaves can use them Thanks,Yuliya From: Adam Bordelon a...@mesosphere.io To: dev@myriad.incubator.apache.org; yuliya Feldman yufeld...@yahoo.com Sent: Tuesday, August 25, 2015 10:06 AM Subject: Re: Question for Mesos gurus Mesos guarantees forward and backward compatibility by one minor version. It is expected that you upgrade the entire cluster to one consecutive version before upgrading any component to the next. So, if your scheduler jar's libmesos is from 0.21.x, you can upgrade your Mesos master/agents to 0.22.x safely, but before you upgrade Mesos to 0.23, you should upgrade your scheduler (and executor) libmesos to 0.22.x. See http://mesos.apache.org/documentation/latest/upgrades/ for other special notes and recommended upgrade order. Once we reach Mesos 1.0 (when the new HTTP API stabilizes), then we'll have stronger guarantees about version compatibility within a major version. On Tue, Aug 25, 2015 at 8:33 AM, yuliya Feldman yufeld...@yahoo.com.invalid wrote: Hello guys, I wonder about compatibility of Mesos protobuf for Myriad usage. If I complied Myriad with Mesos version 0.22.1/0.21.1 but on the cluster I have Mesos 0.23 - is it suppose to be compatible? Yesterday our guys came across an exception(see below). When switching jars to mesos-0.21.1 issue went away. Thanks,Yuliya 15/08/24 10:57:40 INFO scheduler.TaskFactory$NMTaskFactoryImpl: yarn.resourcemanager.hostname is set to rm.marathon.mesos via YARN_RESOURCEMANAGER_OPTS. Passing it into YARN_NODEMANAGER_OPTS. Aug 24, 2015 10:57:40 AM com.lmax.disruptor.FatalExceptionHandler handleEventException SEVERE: Exception processing: 1 com.ebay.myriad.scheduler.event.ResourceOffersEvent@74a1e0a5 java.lang.NoSuchMethodError: org.apache.mesos.Protos$TaskInfo$Builder.setData(Lcom/google/protobuf/ByteString;)Lorg/apache/mesos/Protos$TaskInfo$Builder; at com.ebay.myriad.scheduler.TaskFactory$NMTaskFactoryImpl.createTask(TaskFactory.java:310) at com.ebay.myriad.scheduler.event.handlers.ResourceOffersEventHandler.onEvent(ResourceOffersEventHandler.java:98) at com.ebay.myriad.scheduler.event.handlers.ResourceOffersEventHandler.onEvent(ResourceOffersEventHandler.java:55) at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 15/08/24 10:57:40 ERROR yarn.YarnUncaughtExceptionHandler: Thread Thread[pool-2-thread-3,5,main] threw an Exception. java.lang.RuntimeException: java.lang.NoSuchMethodError: org.apache.mesos.Protos$TaskInfo$Builder.setData(Lcom/google/protobuf/ByteString;)Lorg/apache/mesos/Protos$TaskInfo$Builder; at com.lmax.disruptor.FatalExceptionHandler.handleEventException(FatalExceptionHandler.java:45) at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:147) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NoSuchMethodError: org.apache.mesos.Protos$TaskInfo$Builder.setData(Lcom/google/protobuf/ByteString;)Lorg/apache/mesos/Protos$TaskInfo$Builder; at com.ebay.myriad.scheduler.TaskFactory$NMTaskFactoryImpl.createTask(TaskFactory.java:310) at com.ebay.myriad.scheduler.event.handlers.ResourceOffersEventHandler.onEvent(ResourceOffersEventHandler.java:98) at com.ebay.myriad.scheduler.event.handlers.ResourceOffersEventHandler.onEvent(ResourceOffersEventHandler.java:55) at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128) ... 3 more
Re: Question for Mesos gurus
Yes, you'll have to recompile your scheduler/executor against the latest libmesos. In the upgrade guide, this is mentioned as Upgrade the schedulers by linking the latest native library / jar / egg (if necessary). The modules instructions apply to C++ plugins for the Mesos master/slaves themselves. On Tue, Aug 25, 2015 at 10:52 AM, yuliya Feldman yufeld...@yahoo.com.invalid wrote: Thank you Adam When you say:but before you upgrade Mesos to 0.23, you should upgrade your scheduler (and executor) libmesos to 0.22.x Do you mean - recompile? Does this sentence from link with upgrade instructions you provided means the same? Rebuild and install any modules so that upgraded masters/slaves can use them Thanks,Yuliya From: Adam Bordelon a...@mesosphere.io To: dev@myriad.incubator.apache.org; yuliya Feldman yufeld...@yahoo.com Sent: Tuesday, August 25, 2015 10:06 AM Subject: Re: Question for Mesos gurus Mesos guarantees forward and backward compatibility by one minor version. It is expected that you upgrade the entire cluster to one consecutive version before upgrading any component to the next. So, if your scheduler jar's libmesos is from 0.21.x, you can upgrade your Mesos master/agents to 0.22.x safely, but before you upgrade Mesos to 0.23, you should upgrade your scheduler (and executor) libmesos to 0.22.x. See http://mesos.apache.org/documentation/latest/upgrades/ for other special notes and recommended upgrade order. Once we reach Mesos 1.0 (when the new HTTP API stabilizes), then we'll have stronger guarantees about version compatibility within a major version. On Tue, Aug 25, 2015 at 8:33 AM, yuliya Feldman yufeld...@yahoo.com.invalid wrote: Hello guys, I wonder about compatibility of Mesos protobuf for Myriad usage. If I complied Myriad with Mesos version 0.22.1/0.21.1 but on the cluster I have Mesos 0.23 - is it suppose to be compatible? Yesterday our guys came across an exception(see below). When switching jars to mesos-0.21.1 issue went away. Thanks,Yuliya 15/08/24 10:57:40 INFO scheduler.TaskFactory$NMTaskFactoryImpl: yarn.resourcemanager.hostname is set to rm.marathon.mesos via YARN_RESOURCEMANAGER_OPTS. Passing it into YARN_NODEMANAGER_OPTS. Aug 24, 2015 10:57:40 AM com.lmax.disruptor.FatalExceptionHandler handleEventException SEVERE: Exception processing: 1 com.ebay.myriad.scheduler.event.ResourceOffersEvent@74a1e0a5 java.lang.NoSuchMethodError: org.apache.mesos.Protos$TaskInfo$Builder.setData(Lcom/google/protobuf/ByteString;)Lorg/apache/mesos/Protos$TaskInfo$Builder; at com.ebay.myriad.scheduler.TaskFactory$NMTaskFactoryImpl.createTask(TaskFactory.java:310) at com.ebay.myriad.scheduler.event.handlers.ResourceOffersEventHandler.onEvent(ResourceOffersEventHandler.java:98) at com.ebay.myriad.scheduler.event.handlers.ResourceOffersEventHandler.onEvent(ResourceOffersEventHandler.java:55) at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 15/08/24 10:57:40 ERROR yarn.YarnUncaughtExceptionHandler: Thread Thread[pool-2-thread-3,5,main] threw an Exception. java.lang.RuntimeException: java.lang.NoSuchMethodError: org.apache.mesos.Protos$TaskInfo$Builder.setData(Lcom/google/protobuf/ByteString;)Lorg/apache/mesos/Protos$TaskInfo$Builder; at com.lmax.disruptor.FatalExceptionHandler.handleEventException(FatalExceptionHandler.java:45) at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:147) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NoSuchMethodError: org.apache.mesos.Protos$TaskInfo$Builder.setData(Lcom/google/protobuf/ByteString;)Lorg/apache/mesos/Protos$TaskInfo$Builder; at com.ebay.myriad.scheduler.TaskFactory$NMTaskFactoryImpl.createTask(TaskFactory.java:310) at com.ebay.myriad.scheduler.event.handlers.ResourceOffersEventHandler.onEvent(ResourceOffersEventHandler.java:98) at com.ebay.myriad.scheduler.event.handlers.ResourceOffersEventHandler.onEvent(ResourceOffersEventHandler.java:55) at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128) ... 3 more
Question for Mesos gurus
Hello guys, I wonder about compatibility of Mesos protobuf for Myriad usage. If I complied Myriad with Mesos version 0.22.1/0.21.1 but on the cluster I have Mesos 0.23 - is it suppose to be compatible? Yesterday our guys came across an exception(see below). When switching jars to mesos-0.21.1 issue went away. Thanks,Yuliya 15/08/24 10:57:40 INFO scheduler.TaskFactory$NMTaskFactoryImpl: yarn.resourcemanager.hostname is set to rm.marathon.mesos via YARN_RESOURCEMANAGER_OPTS. Passing it into YARN_NODEMANAGER_OPTS. Aug 24, 2015 10:57:40 AM com.lmax.disruptor.FatalExceptionHandler handleEventException SEVERE: Exception processing: 1 com.ebay.myriad.scheduler.event.ResourceOffersEvent@74a1e0a5 java.lang.NoSuchMethodError: org.apache.mesos.Protos$TaskInfo$Builder.setData(Lcom/google/protobuf/ByteString;)Lorg/apache/mesos/Protos$TaskInfo$Builder; at com.ebay.myriad.scheduler.TaskFactory$NMTaskFactoryImpl.createTask(TaskFactory.java:310) at com.ebay.myriad.scheduler.event.handlers.ResourceOffersEventHandler.onEvent(ResourceOffersEventHandler.java:98) at com.ebay.myriad.scheduler.event.handlers.ResourceOffersEventHandler.onEvent(ResourceOffersEventHandler.java:55) at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 15/08/24 10:57:40 ERROR yarn.YarnUncaughtExceptionHandler: Thread Thread[pool-2-thread-3,5,main] threw an Exception. java.lang.RuntimeException: java.lang.NoSuchMethodError: org.apache.mesos.Protos$TaskInfo$Builder.setData(Lcom/google/protobuf/ByteString;)Lorg/apache/mesos/Protos$TaskInfo$Builder; at com.lmax.disruptor.FatalExceptionHandler.handleEventException(FatalExceptionHandler.java:45) at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:147) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NoSuchMethodError: org.apache.mesos.Protos$TaskInfo$Builder.setData(Lcom/google/protobuf/ByteString;)Lorg/apache/mesos/Protos$TaskInfo$Builder; at com.ebay.myriad.scheduler.TaskFactory$NMTaskFactoryImpl.createTask(TaskFactory.java:310) at com.ebay.myriad.scheduler.event.handlers.ResourceOffersEventHandler.onEvent(ResourceOffersEventHandler.java:98) at com.ebay.myriad.scheduler.event.handlers.ResourceOffersEventHandler.onEvent(ResourceOffersEventHandler.java:55) at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128) ... 3 more
