Dan Smith created GEODE-3172:
--------------------------------
Summary: Clients miss events when servers are upgraded from 1.0 to
1.2
Key: GEODE-3172
URL: https://issues.apache.org/jira/browse/GEODE-3172
Project: Geode
Issue Type: Bug
Components: client queues, membership
Reporter: Dan Smith
We're seeing another data loss issue when upgrading servers due to GEODE-2137.
The issue is that we store HAEventWrapper objects in a region for
server->client queues. These objects contain a member ID. Because the objects
are region values, they can be lazily deserialized using the version of the
current member, which may not match the version of the member they were
serialized with.
What we are seeing is that when a client is creating a queue on a 9.1 server,
and it is copying the contents of the queue from a 9.0 server, we get are
getting serialization errors which prevent the queue from being created. When
the 9.0 server is killed as part of a rolling upgrade, this results in event
loss.
{noformat}
[severe 2017/07/06 15:09:52.195 PDT <Handshaker 0.0.0.0/0.0.0.0:23779 Thread 6>
tid=0xc0] Uncaught exception in thread Thread[Handshaker 0.0.0.0/0.0.0.0:23779
Thread 6,5,Handshaker 0.0.0.0/0.0.0.0:23779]
org.apache.geode.InternalGemFireError: unexpected typeCode: -126
at Remote Member '172.1.1.1(32406)<v4>:32771' in
org.apache.geode.internal.InternalDataSerializer.decodePrimitiveClass(InternalDataSerializer.java:1880)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
org.apache.geode.DataSerializer.readClass(DataSerializer.java:264)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
org.apache.geode.internal.InternalDataSerializer.basicReadObject(InternalDataSerializer.java:2707)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
org.apache.geode.DataSerializer.readObject(DataSerializer.java:2961)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
org.apache.geode.internal.cache.tier.sockets.HAEventWrapper.fromData(HAEventWrapper.java:330)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
org.apache.geode.internal.InternalDataSerializer.invokeFromData(InternalDataSerializer.java:2370)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
org.apache.geode.internal.DSFIDFactory.create(DSFIDFactory.java:981)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
org.apache.geode.internal.InternalDataSerializer.basicReadObject(InternalDataSerializer.java:2691)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
org.apache.geode.DataSerializer.readObject(DataSerializer.java:2961)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
org.apache.geode.internal.util.BlobHelper.deserializeBlob(BlobHelper.java:99)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
org.apache.geode.internal.cache.EntryEventImpl.deserialize(EntryEventImpl.java:1911)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
org.apache.geode.internal.cache.EntryEventImpl.deserialize(EntryEventImpl.java:1904)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
org.apache.geode.internal.cache.VMCachedDeserializable.getDeserializedValue(VMCachedDeserializable.java:134)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
org.apache.geode.internal.cache.AbstractRegionMap.initialImagePut(AbstractRegionMap.java:771)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
org.apache.geode.internal.cache.InitialImageOperation.processChunk(InitialImageOperation.java:928)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
org.apache.geode.internal.cache.InitialImageOperation$ImageProcessor.process(InitialImageOperation.java:1249)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
org.apache.geode.distributed.internal.ReplyMessage.process(ReplyMessage.java:213)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
org.apache.geode.internal.cache.InitialImageOperation$ImageReplyMessage.process(InitialImageOperation.java:2723)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
org.apache.geode.distributed.internal.ReplyMessage.dmProcess(ReplyMessage.java:193)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
org.apache.geode.distributed.internal.ReplyMessage.process(ReplyMessage.java:186)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
org.apache.geode.distributed.internal.DistributionMessage.scheduleAction(DistributionMessage.java:374)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
org.apache.geode.distributed.internal.DistributionMessage$1.run(DistributionMessage.java:440)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
org.apache.geode.distributed.internal.DistributionManager.runUntilShutdown(DistributionManager.java:665)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
org.apache.geode.distributed.internal.DistributionManager$5$1.run(DistributionManager.java:961)
at Remote Member '172.1.1.1(32406)<v4>:32771' in
java.lang.Thread.run(Thread.java:748)
at
org.apache.geode.distributed.internal.ReplyException.handleAsUnexpected(ReplyException.java:89)
at
org.apache.geode.internal.cache.InitialImageOperation.getFromOne(InitialImageOperation.java:501)
at
org.apache.geode.internal.cache.DistributedRegion.getInitialImageAndRecovery(DistributedRegion.java:1182)
at
org.apache.geode.internal.cache.DistributedRegion.initialize(DistributedRegion.java:1022)
at
org.apache.geode.internal.cache.HARegion.initialize(HARegion.java:337)
at
org.apache.geode.internal.cache.GemFireCacheImpl.createVMRegion(GemFireCacheImpl.java:3077)
at
org.apache.geode.internal.cache.HARegion.getInstance(HARegion.java:257)
at
org.apache.geode.internal.cache.ha.HARegionQueue.createHARegion(HARegionQueue.java:367)
at
org.apache.geode.internal.cache.ha.HARegionQueue.<init>(HARegionQueue.java:347)
at
org.apache.geode.internal.cache.ha.HARegionQueue$BlockingHARegionQueue.<init>(HARegionQueue.java:2100)
at
org.apache.geode.internal.cache.ha.HARegionQueue$DurableHARegionQueue.<init>(HARegionQueue.java:2331)
at
org.apache.geode.internal.cache.ha.HARegionQueue.getHARegionQueueInstance(HARegionQueue.java:1953)
at
org.apache.geode.internal.cache.tier.sockets.CacheClientProxy$MessageDispatcher.<init>(CacheClientProxy.java:2321)
at
org.apache.geode.internal.cache.tier.sockets.CacheClientProxy.initializeMessageDispatcher(CacheClientProxy.java:1730)
at
org.apache.geode.internal.cache.tier.sockets.CacheClientNotifier.initializeProxy(CacheClientNotifier.java:610)
at
org.apache.geode.internal.cache.tier.sockets.CacheClientNotifier.registerClient(CacheClientNotifier.java:537)
at
org.apache.geode.internal.cache.tier.sockets.CacheClientNotifier.registerGFEClient(CacheClientNotifier.java:330)
at
org.apache.geode.internal.cache.tier.sockets.CacheClientNotifier.registerClient(CacheClientNotifier.java:277)
at
org.apache.geode.internal.cache.tier.sockets.AcceptorImpl.handleNewClientConnection(AcceptorImpl.java:1457)
at
org.apache.geode.internal.cache.tier.sockets.AcceptorImpl$5.run(AcceptorImpl.java:1332)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
{noformat}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
