[
https://issues.apache.org/jira/browse/FLINK-15249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16998036#comment-16998036
]
Chongchen Chen commented on FLINK-15249:
----------------------------------------
[~zhuzh] I construct a test case. On master, the cost is 4300ms on my
computer, with union set, the cost is 2496ms on my computer.
{code:java}
@Test
public void testRegionFailoverPerformance() throws Exception {
TestFailoverTopology.Builder topologyBuilder = new
TestFailoverTopology.Builder();
ArrayList<ArrayList<TestFailoverTopology.TestFailoverVertex>> graph = new
ArrayList<>();
int level = 16;
int curLevel = 0;
for (int i = level; i >= 0; i--, curLevel++) {
ArrayList<TestFailoverTopology.TestFailoverVertex> jv = new ArrayList<>();
int currentNum = 1 << i;
for (int j = 0; j < currentNum; j++) {
TestFailoverTopology.TestFailoverVertex nv =
topologyBuilder.newVertex();
jv.add(nv);
if (curLevel != 0) {
topologyBuilder.connect(graph.get(curLevel - 1).get(j << 1), nv,
ResultPartitionType.PIPELINED);
topologyBuilder.connect(graph.get(curLevel - 1).get((j << 1) + 1),
nv, ResultPartitionType.PIPELINED);
}
}
graph.add(jv);
}
TestFailoverTopology topology = topologyBuilder.build();
TestResultPartitionAvailabilityChecker availabilityChecker = new
TestResultPartitionAvailabilityChecker();
long time = System.currentTimeMillis();
for (int i = 0; i < 10; i++) {
RestartPipelinedRegionStrategy strategy = new
RestartPipelinedRegionStrategy(topology, availabilityChecker);
}
System.out.println("cost:" + (System.currentTimeMillis() - time));
}
{code}
> Improve PipelinedRegions calculation with Union Set
> ---------------------------------------------------
>
> Key: FLINK-15249
> URL: https://issues.apache.org/jira/browse/FLINK-15249
> Project: Flink
> Issue Type: Improvement
> Reporter: Chongchen Chen
> Priority: Major
> Labels: pull-request-available
> Attachments: PipelinedRegionComputeUtil.diff
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Union Set's Merge Set cost is O(1). current implementation is O(N). the
> attachment is patch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
