dcapwell commented on code in PR #50:
URL: https://github.com/apache/cassandra-accord/pull/50#discussion_r1284878013
##########
accord-core/src/main/java/accord/local/CommandStores.java:
##########
@@ -474,40 +474,42 @@ public <O> void mapReduceConsume(PreLoadContext context,
IntStream commandStore
public <O> AsyncChain<O> mapReduce(PreLoadContext context, Routables<?>
keys, long minEpoch, long maxEpoch, MapReduce<? super SafeCommandStore, O>
mapReduce)
{
- AsyncChain<O> chain = null;
BiFunction<O, O, O> reducer = mapReduce::reduce;
Snapshot snapshot = current;
ShardHolder[] shards = snapshot.shards;
+ List<AsyncChain<? extends O>> chains = null;
Review Comment:
feedback from Ariel. There is a bug with combiner where we double called the
callback, because the callback threw. I added a TODO documenting this bug in
the original feedback PR and he asked me to just go and fix it...
All / Reduce / ReduceWithIdentiy were all acting like .map logic, so rather
than making everything handle exceptions properly it was simpler to centralize
exception handling and replace all 3 cases with a .map call; this makes Reduce
class no longer exist.
So for these call paths, had to move the List<AsyncChain<? extends O>>
chains from Reduce to the caller.
##########
accord-core/src/main/java/accord/local/CommandStores.java:
##########
@@ -474,40 +474,42 @@ public <O> void mapReduceConsume(PreLoadContext context,
IntStream commandStore
public <O> AsyncChain<O> mapReduce(PreLoadContext context, Routables<?>
keys, long minEpoch, long maxEpoch, MapReduce<? super SafeCommandStore, O>
mapReduce)
{
- AsyncChain<O> chain = null;
BiFunction<O, O, O> reducer = mapReduce::reduce;
Snapshot snapshot = current;
ShardHolder[] shards = snapshot.shards;
+ List<AsyncChain<? extends O>> chains = null;
for (ShardHolder shard : shards)
{
// TODO (urgent, efficiency): range map for intersecting ranges
(e.g. that to be introduced for range dependencies)
Ranges shardRanges = shard.ranges().allBetween(minEpoch, maxEpoch);
if (!shardRanges.intersects(keys))
continue;
- AsyncChain<O> next = shard.store.submit(context, mapReduce);
- chain = chain != null ? AsyncChains.reduce(chain, next, reducer) :
next;
+ if (chains == null)
+ chains = new ArrayList<>();
+ chains.add(shard.store.submit(context, mapReduce));
}
- if (chain == null)
- return AsyncChains.success(null);
- return chain;
+ return chains == null ?
+ AsyncChains.success(null) :
+ AsyncChains.reduce(chains, reducer);
}
protected <O> AsyncChain<O> mapReduce(PreLoadContext context, IntStream
commandStoreIds, MapReduce<? super SafeCommandStore, O> mapReduce)
{
// TODO (low priority, efficiency): avoid using an array, or use a
scratch buffer
int[] ids = commandStoreIds.toArray();
- AsyncChain<O> chain = null;
+ List<AsyncChain<? extends O>> chains = null;
Review Comment:
see https://github.com/apache/cassandra-accord/pull/50#discussion_r1284878013
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
