imay commented on a change in pull request #851: Parallel fragment exec instance
URL: https://github.com/apache/incubator-doris/pull/851#discussion_r270889524
##########
File path: fe/src/main/java/org/apache/doris/qe/Coordinator.java
##########
@@ -882,12 +889,60 @@ private void computeFragmentHosts() throws Exception {
}
}
- //One fragment could only have one HashJoinNode
- private boolean isColocateJoin(PlanNode node) {
- if (Config.disable_colocate_join) {
+ private boolean needParallelInstance(PlanNode leftMostNode) {
+ if (getParallelExecInstanceNum() <=
SessionVariable.MIN_EXEC_INSTANCE_NUM) {
return false;
}
+ if (leftMostNode instanceof OlapScanNode) {
+ OlapScanNode olapScanNode = (OlapScanNode) leftMostNode;
+
+ //case 1: the small table of broadcast join need not parallel
+ PlanFragment destFragment =
olapScanNode.getFragment().getDestFragment();
+ if (destFragment != null) {
+ PlanNode destRootNode = destFragment.getPlanRoot();
+ if (destRootNode instanceof HashJoinNode) {
+ HashJoinNode joinNode = (HashJoinNode) destRootNode;
+ if (!joinNode.isShuffleJoin()) {
+ LOG.debug("ScanNode {} for fragment {} need not
parallel because of broadcast join",
+ olapScanNode.getOlapTable().getName(),
olapScanNode.getFragmentId());
+ return false;
+ }
+ }
Review comment:
For simple, I think we can parallel for all queries.
However one thing we should notice is that when we enable fragment parallel,
FE would send more `PlanFragmentInstance` than before, which would cost FE more
CPU to serialize requests and more network bandwidth to transfer requests. Then
as the number of concurrent queries increase, there would be some bottleneck
for FE.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
