[jira] [Commented] (GEODE-4704) GatewaySender batch conflation can incorrectly conflate events causing out of order processing
[ https://issues.apache.org/jira/browse/GEODE-4704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16379000#comment-16379000 ] ASF subversion and git services commented on GEODE-4704: Commit 147e6bc6b9009aa8e878141ef30e55145732943d in geode's branch refs/heads/feature/GEODE-4685 from [~barry.oglesby] [ https://gitbox.apache.org/repos/asf?p=geode.git;h=147e6bc ] GEODE-4704: Modified ConflationKey to use shadowKey when comparing events > GatewaySender batch conflation can incorrectly conflate events causing out of > order processing > -- > > Key: GEODE-4704 > URL: https://issues.apache.org/jira/browse/GEODE-4704 > Project: Geode > Issue Type: Bug > Components: wan >Reporter: Barry Oglesby >Priority: Major > Labels: pull-request-available > Fix For: 1.5.0 > > Time Spent: 20m > Remaining Estimate: 0h > > In the current implementation of \{{AbstractGatewaySenderEventProcessor > conflate}} can incorrectly conflate two create events on the same key. > The else clause of the conflate method (meaning the event is not conflatable) > assumes there isn't already a event with the same region, key and operation > in the map. It just does a put: > {noformat} > ConflationKey key = new ConflationKey(gsEvent.getRegion().getFullPath(), > gsEvent.getKeyToConflate(), gsEvent.getOperation()); > conflatedEventsMap.put(key, gsEvent); > {noformat} > In certain cases, there could already be a create event on the same region, > key and operation in the batch, so the previous one gets replaced by the > later once which causes the events to be processed out of order. > These 4 events: > {noformat} > SenderEventImpl[id=EventIDid=57bytes;threadID=0x10018|112;sequenceID=100;bucketId=24];operation=CREATE;region=/dataStoreRegion;key=Object_13964;shadowKey=27709] > SenderEventImpl[id=EventIDid=57bytes;threadID=0x10018|112;sequenceID=101;bucketId=24];operation=CREATE;region=/dataStoreRegion;key=Object_14024;shadowKey=27822] > SenderEventImpl[id=EventIDid=57bytes;threadID=0x10018|112;sequenceID=102;bucketId=24];operation=DESTROY;region=/dataStoreRegion;key=Object_13964;shadowKey=27935] > SenderEventImpl[id=EventIDid=57bytes;threadID=0x10018|112;sequenceID=104;bucketId=24];operation=CREATE;region=/dataStoreRegion;key=Object_14024;shadowKey=28161] > {noformat} > Become these 3 events: > {noformat} > SenderEventImpl[id=EventIDid=57bytes;threadID=0x10018|112;sequenceID=100;bucketId=24];operation=CREATE;region=/dataStoreRegion;key=Object_13964;shadowKey=27709] > SenderEventImpl[id=EventIDid=57bytes;threadID=0x10018|112;sequenceID=104;bucketId=24];operation=CREATE;region=/dataStoreRegion;key=Object_14024;shadowKey=28161] > SenderEventImpl[id=EventIDid=57bytes;threadID=0x10018|112;sequenceID=102;bucketId=24];operation=DESTROY;region=/dataStoreRegion;key=Object_13964;shadowKey=27935] > {noformat} > Notice the shadowKeys and sequenceIds are out of order after the conflation. > The fix will be to include the shadow key in hashCode and equals in the case > where the event is not conflatable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (GEODE-4704) GatewaySender batch conflation can incorrectly conflate events causing out of order processing
[ https://issues.apache.org/jira/browse/GEODE-4704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377241#comment-16377241 ] ASF subversion and git services commented on GEODE-4704: Commit 147e6bc6b9009aa8e878141ef30e55145732943d in geode's branch refs/heads/feature/GEODE-4738 from [~barry.oglesby] [ https://gitbox.apache.org/repos/asf?p=geode.git;h=147e6bc ] GEODE-4704: Modified ConflationKey to use shadowKey when comparing events > GatewaySender batch conflation can incorrectly conflate events causing out of > order processing > -- > > Key: GEODE-4704 > URL: https://issues.apache.org/jira/browse/GEODE-4704 > Project: Geode > Issue Type: Bug > Components: wan >Reporter: Barry Oglesby >Priority: Major > Labels: pull-request-available > Fix For: 1.5.0 > > Time Spent: 20m > Remaining Estimate: 0h > > In the current implementation of \{{AbstractGatewaySenderEventProcessor > conflate}} can incorrectly conflate two create events on the same key. > The else clause of the conflate method (meaning the event is not conflatable) > assumes there isn't already a event with the same region, key and operation > in the map. It just does a put: > {noformat} > ConflationKey key = new ConflationKey(gsEvent.getRegion().getFullPath(), > gsEvent.getKeyToConflate(), gsEvent.getOperation()); > conflatedEventsMap.put(key, gsEvent); > {noformat} > In certain cases, there could already be a create event on the same region, > key and operation in the batch, so the previous one gets replaced by the > later once which causes the events to be processed out of order. > These 4 events: > {noformat} > SenderEventImpl[id=EventIDid=57bytes;threadID=0x10018|112;sequenceID=100;bucketId=24];operation=CREATE;region=/dataStoreRegion;key=Object_13964;shadowKey=27709] > SenderEventImpl[id=EventIDid=57bytes;threadID=0x10018|112;sequenceID=101;bucketId=24];operation=CREATE;region=/dataStoreRegion;key=Object_14024;shadowKey=27822] > SenderEventImpl[id=EventIDid=57bytes;threadID=0x10018|112;sequenceID=102;bucketId=24];operation=DESTROY;region=/dataStoreRegion;key=Object_13964;shadowKey=27935] > SenderEventImpl[id=EventIDid=57bytes;threadID=0x10018|112;sequenceID=104;bucketId=24];operation=CREATE;region=/dataStoreRegion;key=Object_14024;shadowKey=28161] > {noformat} > Become these 3 events: > {noformat} > SenderEventImpl[id=EventIDid=57bytes;threadID=0x10018|112;sequenceID=100;bucketId=24];operation=CREATE;region=/dataStoreRegion;key=Object_13964;shadowKey=27709] > SenderEventImpl[id=EventIDid=57bytes;threadID=0x10018|112;sequenceID=104;bucketId=24];operation=CREATE;region=/dataStoreRegion;key=Object_14024;shadowKey=28161] > SenderEventImpl[id=EventIDid=57bytes;threadID=0x10018|112;sequenceID=102;bucketId=24];operation=DESTROY;region=/dataStoreRegion;key=Object_13964;shadowKey=27935] > {noformat} > Notice the shadowKeys and sequenceIds are out of order after the conflation. > The fix will be to include the shadow key in hashCode and equals in the case > where the event is not conflatable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (GEODE-4704) GatewaySender batch conflation can incorrectly conflate events causing out of order processing
[ https://issues.apache.org/jira/browse/GEODE-4704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16375122#comment-16375122 ] ASF subversion and git services commented on GEODE-4704: Commit 147e6bc6b9009aa8e878141ef30e55145732943d in geode's branch refs/heads/develop from [~barry.oglesby] [ https://gitbox.apache.org/repos/asf?p=geode.git;h=147e6bc ] GEODE-4704: Modified ConflationKey to use shadowKey when comparing events > GatewaySender batch conflation can incorrectly conflate events causing out of > order processing > -- > > Key: GEODE-4704 > URL: https://issues.apache.org/jira/browse/GEODE-4704 > Project: Geode > Issue Type: Bug > Components: wan >Reporter: Barry Oglesby >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > In the current implementation of \{{AbstractGatewaySenderEventProcessor > conflate}} can incorrectly conflate two create events on the same key. > The else clause of the conflate method (meaning the event is not conflatable) > assumes there isn't already a event with the same region, key and operation > in the map. It just does a put: > {noformat} > ConflationKey key = new ConflationKey(gsEvent.getRegion().getFullPath(), > gsEvent.getKeyToConflate(), gsEvent.getOperation()); > conflatedEventsMap.put(key, gsEvent); > {noformat} > In certain cases, there could already be a create event on the same region, > key and operation in the batch, so the previous one gets replaced by the > later once which causes the events to be processed out of order. > These 4 events: > {noformat} > SenderEventImpl[id=EventIDid=57bytes;threadID=0x10018|112;sequenceID=100;bucketId=24];operation=CREATE;region=/dataStoreRegion;key=Object_13964;shadowKey=27709] > SenderEventImpl[id=EventIDid=57bytes;threadID=0x10018|112;sequenceID=101;bucketId=24];operation=CREATE;region=/dataStoreRegion;key=Object_14024;shadowKey=27822] > SenderEventImpl[id=EventIDid=57bytes;threadID=0x10018|112;sequenceID=102;bucketId=24];operation=DESTROY;region=/dataStoreRegion;key=Object_13964;shadowKey=27935] > SenderEventImpl[id=EventIDid=57bytes;threadID=0x10018|112;sequenceID=104;bucketId=24];operation=CREATE;region=/dataStoreRegion;key=Object_14024;shadowKey=28161] > {noformat} > Become these 3 events: > {noformat} > SenderEventImpl[id=EventIDid=57bytes;threadID=0x10018|112;sequenceID=100;bucketId=24];operation=CREATE;region=/dataStoreRegion;key=Object_13964;shadowKey=27709] > SenderEventImpl[id=EventIDid=57bytes;threadID=0x10018|112;sequenceID=104;bucketId=24];operation=CREATE;region=/dataStoreRegion;key=Object_14024;shadowKey=28161] > SenderEventImpl[id=EventIDid=57bytes;threadID=0x10018|112;sequenceID=102;bucketId=24];operation=DESTROY;region=/dataStoreRegion;key=Object_13964;shadowKey=27935] > {noformat} > Notice the shadowKeys and sequenceIds are out of order after the conflation. > The fix will be to include the shadow key in hashCode and equals in the case > where the event is not conflatable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)