imay commented on a change in pull request #851: Parallel fragment exec instance
URL: https://github.com/apache/incubator-doris/pull/851#discussion_r270853216
##########
File path: fe/src/main/java/org/apache/doris/qe/Coordinator.java
##########
@@ -882,12 +889,60 @@ private void computeFragmentHosts() throws Exception {
}
}
- //One fragment could only have one HashJoinNode
- private boolean isColocateJoin(PlanNode node) {
- if (Config.disable_colocate_join) {
+ private boolean needParallelInstance(PlanNode leftMostNode) {
+ if (getParallelExecInstanceNum() <=
SessionVariable.MIN_EXEC_INSTANCE_NUM) {
return false;
}
+ if (leftMostNode instanceof OlapScanNode) {
+ OlapScanNode olapScanNode = (OlapScanNode) leftMostNode;
+
+ //case 1: the small table of broadcast join need not parallel
+ PlanFragment destFragment =
olapScanNode.getFragment().getDestFragment();
+ if (destFragment != null) {
+ PlanNode destRootNode = destFragment.getPlanRoot();
+ if (destRootNode instanceof HashJoinNode) {
+ HashJoinNode joinNode = (HashJoinNode) destRootNode;
+ if (!joinNode.isShuffleJoin()) {
+ LOG.debug("ScanNode {} for fragment {} need not
parallel because of broadcast join",
+ olapScanNode.getOlapTable().getName(),
olapScanNode.getFragmentId());
+ return false;
+ }
+ }
Review comment:
Does this works? If the fragment is agg(hash(scan1, scan2)), for scan2, the
RootNode of `destFragment` is `agg`.
I think if it isn't easy to match this case, we can return true for it.
For broadcast join, I think the bigger influence is that the outer child
will be parallel, and making it would need more hash table for inner child. So
it's ok to scan parallel here
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
