dferstay commented on a change in pull request #694:
URL: https://github.com/apache/pulsar-client-go/pull/694#discussion_r777248480
##########
File path: pulsar/default_router.go
##########
@@ -80,32 +79,27 @@ func NewDefaultRouter(
// spread the data on different partitions but not necessarily
in a specific sequence.
var now int64
size := uint32(len(message.Payload))
- previousMessageCount := atomic.LoadUint32(&state.msgCounter)
- previousBatchingMaxSize :=
atomic.LoadUint32(&state.cumulativeBatchSize)
- previousLastChange :=
atomic.LoadInt64(&state.lastChangeTimestamp)
+ messageCount := atomic.AddUint32(&state.msgCounter, 1)
+ batchSize := atomic.AddUint32(&state.cumulativeBatchSize, size)
- messageCountReached := previousMessageCount >=
uint32(maxBatchingMessages-1)
- sizeReached := (size >=
uint32(maxBatchingSize)-previousBatchingMaxSize)
+ messageCountReached := messageCount%uint32(maxBatchingMessages)
== 0
Review comment:
I've made a refinement that uses compare-and-swap to keep multiple
go-routines racing to increment the partition cursor which was causing
partitions to be skipped. This has the side-effect that only one of the racing
go-routines will read the system clock to record the time the last batch was
completed. Results before and after this commit are below:
```
name old time/op new time/op delta
DefaultRouterParallel 14.7ns ± 1% 18.5ns ± 4% +25.88% (p=0.000 n=9+9)
DefaultRouterParallel-2 55.0ns ±13% 43.5ns ± 2% -20.93% (p=0.000 n=10+8)
DefaultRouterParallel-4 53.5ns ± 9% 40.0ns ±11% -25.29% (p=0.000
n=10+10)
DefaultRouterParallel-8 54.2ns ± 8% 41.9ns ± 5% -22.68% (p=0.000
n=10+10)
DefaultRouterParallel-16 56.4ns ±21% 41.7ns ±22% -25.98% (p=0.000
n=10+10)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
