We just started using 0.5, so It was a non-prod environment, and we reloaded the data. (I¹m sure that is no help at all)
Definitely check your classpath for the different kryo versions though. Titan has a kryo as a dependency and it conflicts with Storm¹s. The classpath may be different when running in the cluster. (I believe it is possible to capture the line from the log file that is spawning the worker to look at the classpath) -brian --- Brian O'Neill Chief Technology Officer Health Market Science The Science of Better Results 2700 Horizon Drive King of Prussia, PA 19406 M: 215.588.6024 @boneill42 <http://www.twitter.com/boneill42> healthmarketscience.com This information transmitted in this email message is for the intended recipient only and may contain confidential and/or privileged material. If you received this email in error and are not the intended recipient, or the person responsible to deliver it to the intended recipient, please contact the sender at the email above and delete this email and any attachments and destroy any copies thereof. Any review, retransmission, dissemination, copying or other use of, or taking any action in reliance upon, this information by persons or entities other than the intended recipient is strictly prohibited. From: Nick Beenham <[email protected]> Reply-To: <[email protected]> Date: Thursday, October 9, 2014 at 3:45 PM To: <[email protected]> Subject: Re: What appears to be a Kryo Conflict It was stored with 0.5 and is being read with the same version. what was your solution? thanks, Nick On Thu, Oct 9, 2014 at 3:36 PM, Brian O'Neill <[email protected]> wrote: > > We ran into the same problem. > > What version of Titan was used to load the graph database? > > If you¹ve stored data with 0.4, and you are trying to read it with 0.5 (from > Storm), you may run into Kryo version issues. > > -brian > > --- > Brian O'Neill > Chief Technology Officer > > > Health Market Science > The Science of Better Results > 2700 Horizon Drive King of Prussia, PA 19406 > M: 215.588.6024 <tel:215.588.6024> @boneill42 > <http://www.twitter.com/boneill42> > healthmarketscience.com > > > This information transmitted in this email message is for the intended > recipient only and may contain confidential and/or privileged material. If you > received this email in error and are not the intended recipient, or the person > responsible to deliver it to the intended recipient, please contact the sender > at the email above and delete this email and any attachments and destroy any > copies thereof. Any review, retransmission, dissemination, copying or other > use of, or taking any action in reliance upon, this information by persons or > entities other than the intended recipient is strictly prohibited. > > > > From: Nick Beenham <[email protected]> > Reply-To: <[email protected]> > Date: Thursday, October 9, 2014 at 2:58 PM > To: <[email protected]> > Subject: What appears to be a Kryo Conflict > > Hi all, > > I have a peculiar problem... > > I'm using a kafka storm combo to feed updates to my Titan Graph db. The > topology runs quite happily when local but as soon as I try to run it on a > cluster I run into Kryo errors. > > 2014-10-09 18:41:47 b.s.util [ERROR] Async loop died! > java.lang.RuntimeException: com.esotericsoftware.kryo.KryoException: > Encountered unregistered class ID: 10 > at > backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:1 > 28) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > at > backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.j > ava:99) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > at > backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > at > backtype.storm.daemon.executor$fn__5641$fn__5653$fn__5700.invoke(executor.clj: > 746) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > at backtype.storm.util$async_loop$fn__457.invoke(util.clj:431) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na] > at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45] > Caused by: com.esotericsoftware.kryo.KryoException: Encountered unregistered > class ID: 10 > at > com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassReso > lver.java:119) ~[kryo-2.21.jar:na] > at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:610) ~[kryo-2.21.jar:na] > at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:721) > ~[kryo-2.21.jar:na] > at > com.thinkaurelius.titan.graphdb.database.serialize.kryo.KryoSerializer.readCla > ssAndObject(KryoSerializer.java:77) ~[stormjar.jar:na] > at > com.thinkaurelius.titan.graphdb.database.serialize.StandardSerializer.readClas > sAndObject(StandardSerializer.java:85) ~[stormjar.jar:na] > at > com.thinkaurelius.titan.graphdb.database.EdgeSerializer.readPropertyValue(Edge > Serializer.java:206) ~[stormjar.jar:na] > at > com.thinkaurelius.titan.graphdb.database.EdgeSerializer.readPropertyValue(Edge > Serializer.java:201) ~[stormjar.jar:na] > at > com.thinkaurelius.titan.graphdb.database.EdgeSerializer.parseRelation(EdgeSeri > alizer.java:134) ~[stormjar.jar:na] > at > com.thinkaurelius.titan.graphdb.database.EdgeSerializer.readRelation(EdgeSeria > lizer.java:70) ~[stormjar.jar:na] > at > com.thinkaurelius.titan.graphdb.transaction.RelationConstructor.readRelation(R > elationConstructor.java:61) ~[stormjar.jar:na] > at > com.thinkaurelius.titan.graphdb.transaction.RelationConstructor$1$1.next(Relat > ionConstructor.java:46) ~[stormjar.jar:na] > at > com.thinkaurelius.titan.graphdb.transaction.RelationConstructor$1$1.next(Relat > ionConstructor.java:34) ~[stormjar.jar:na] > at > com.thinkaurelius.titan.graphdb.types.vertices.TitanSchemaVertex.getDefinition > (TitanSchemaVertex.java:73) ~[stormjar.jar:na] > at > com.thinkaurelius.titan.graphdb.types.vertices.PropertyKeyVertex.getDataType(P > ropertyKeyVertex.java:17) ~[stormjar.jar:na] > at > com.thinkaurelius.titan.graphdb.query.QueryUtil.constraints2QNF(QueryUtil.java > :148) ~[stormjar.jar:na] > at > com.thinkaurelius.titan.graphdb.query.graph.GraphCentricQueryBuilder.construct > Query(GraphCentricQueryBuilder.java:208) ~[stormjar.jar:na] > at > com.thinkaurelius.titan.graphdb.query.graph.GraphCentricQueryBuilder.vertices( > GraphCentricQueryBuilder.java:155) ~[stormjar.jar:na] > at > com.tinkerpop.pipes.transform.GraphQueryPipe.processNextStart(GraphQueryPipe.j > ava:49) ~[stormjar.jar:na] > at > com.tinkerpop.pipes.transform.GraphQueryPipe.processNextStart(GraphQueryPipe.j > ava:17) ~[stormjar.jar:na] > at com.tinkerpop.pipes.AbstractPipe.next(AbstractPipe.java:89) > ~[stormjar.jar:na] > at com.tinkerpop.pipes.IdentityPipe.processNextStart(IdentityPipe.java:19) > ~[stormjar.jar:na] > at com.tinkerpop.pipes.AbstractPipe.next(AbstractPipe.java:89) > ~[stormjar.jar:na] > at com.tinkerpop.pipes.util.Pipeline.next(Pipeline.java:115) > ~[stormjar.jar:na] > at com.tinkerpop.pipes.util.PipeHelper.fillCollection(PipeHelper.java:52) > ~[stormjar.jar:na] > at > com.tinkerpop.gremlin.java.GremlinPipeline.toList(GremlinPipeline.java:1564) > ~[stormjar.jar:na] > at > com.comcast.graph.event.bolt.GraphSearchMetricsBolt.execute(GraphSearchMetrics > Bolt.java:83) ~[stormjar.jar:na] > at > backtype.storm.topology.BasicBoltExecutor.execute(BasicBoltExecutor.java:50) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > at > backtype.storm.daemon.executor$fn__5641$tuple_action_fn__5643.invoke(executor. > clj:631) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > at > backtype.storm.daemon.executor$mk_task_receiver$fn__5564.invoke(executor.clj:3 > 99) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > at > backtype.storm.disruptor$clojure_handler$reify__745.onEvent(disruptor.clj:58) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > at > backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:1 > 20) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > ... 6 common frames omitted > > Looking at my dependency tree, titan has a dependency on kryo 2.22, how does > this conflict with Storm and 2.21? > > titan: > [INFO] | \- com.esotericsoftware.kryo:kryo:jar:2.22:compile > > Regards, > > Nick >
