github-actions[bot] commented on code in PR #69923:
URL: https://github.com/apache/doris/pull/69923#discussion_r4435324212
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/glue/translator/PhysicalPlanTranslator.java:
##########
@@ -1046,11 +1046,17 @@ private PlanFragment
computePhysicalOlapScan(PhysicalOlapScan olapScan, PlanTran
.map(context::findSlotRef).collect(Collectors.toList());
dataPartition = new DataPartition(TPartitionType.HASH_PARTITIONED,
partitionExprs);
}
+ if (olapScan.getDistributionSpec() instanceof
DistributionSpecStorageGather) {
+ dataPartition = DataPartition.UNPARTITIONED;
Review Comment:
[P1] Keep a real exchange before fixed-backend local TVF sinks
With effective DOP=1 and query cache disabled, a nonempty one-tablet source
can now produce this reduced plan for `INSERT INTO
local('file_path'='/tmp/out_', 'backend_id'='<alive BE>', 'format'='csv')
SELECT count(case when cast(v as boolean) then 1 end), sum(v) FROM t`:
```text
TVFTableSink(local, backend_id=B)
GlobalScalarAggregate(conditional_count, sum(v))
OlapScan(t, StorageGather)
```
`StorageGather` satisfies the TVF sink's `GATHER` request, so
`visitPhysicalTVFTableSink` attaches the sink directly to this scan fragment.
`UnassignedJobBuilder.buildJob` then chooses `UnassignedLocalTVFSinkJob` before
ordinary scan assignment; that job supplies `DefaultScanSource(empty map)`.
`ThriftPlansBuilder` therefore sends no scan ranges, and
`OlapScanLocalState::_init_scanners` returns successful EOS. The export
computes `0,NULL` over empty input instead of the source values, even when B
holds its replica. Preserve an actual relocation exchange at this fixed-worker
boundary (or make its scheduler correctly retain the scan input), and cover a
nonempty one-tablet local export.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/implementation/LogicalOlapScanToPhysicalOlapScan.java:
##########
@@ -46,34 +50,70 @@
public class LogicalOlapScanToPhysicalOlapScan extends
OneImplementationRuleFactory {
@Override
public Rule build() {
- return logicalOlapScan().then(olapScan ->
- new PhysicalOlapScan(
- olapScan.getRelationId(),
- olapScan.getTable(),
- olapScan.getQualifier(),
- olapScan.getSelectedIndexId(),
- olapScan.getSelectedTabletIds(),
- olapScan.getSelectedPartitionIds(),
- olapScan.hasPartitionPredicate(),
- convertDistribution(olapScan),
- olapScan.getPreAggStatus(),
-
olapScan.getOutputByIndex(olapScan.getTable().getBaseIndexId()),
- Optional.empty(),
- olapScan.getLogicalProperties(),
- null,
- null,
- olapScan.getTableSample(),
- olapScan.getOperativeSlots(),
- olapScan.getVirtualColumns(),
- olapScan.getScoreOrderKeys(),
- olapScan.getScoreLimit(),
- olapScan.getScoreRangeInfo(),
- olapScan.getAnnOrderKeys(),
- olapScan.getAnnLimit(),
- olapScan.getTableAlias(),
- olapScan.getPartitionPrunablePredicates(),
- olapScan.getScanParams())
- ).toRule(RuleType.LOGICAL_OLAP_SCAN_TO_PHYSICAL_OLAP_SCAN_RULE);
+ return logicalOlapScan().thenApplyMulti(ctx -> {
+ LogicalOlapScan olapScan = ctx.root;
+ DistributionSpec distribution = convertDistribution(olapScan);
+ if
(SessionVariable.canUseNereidsDistributePlanner(ctx.connectContext)
+ && canUseSingleTabletGather(olapScan,
ctx.connectContext.getSessionVariable(),
+
ctx.connectContext.getSessionVariable().resolveCloudClusterName(ctx.connectContext)))
{
+ // Keep the natural distribution candidate for
colocate/bucket-shuffle plans.
+ return ImmutableList.of(implement(olapScan, distribution),
+ implement(olapScan,
DistributionSpecStorageGather.INSTANCE));
Review Comment:
[P1] Make forced gathers select an actual exchange for NLJ inputs
With effective DOP=1, cache disabled and two nonempty single-tablet tables,
`SELECT a.k,b.k FROM a FULL OUTER JOIN b ON a.k < b.k` has this reduced
candidate:
```text
NestedLoopJoin(FULL_OUTER, a.k < b.k)
OlapScan(a, StorageGather)
OlapScan(b, StorageGather)
```
`RequestPropertyDeriver.visitPhysicalNestedLoopJoin` requires `GATHER` on
both sides. `CostAndEnforcerJob.enforce` records each cheaper StorageGather
scan as `best[GATHER]`. The regulator's existing `updateChildEnforceAndCost(1,
GATHER)` cannot force the right exchange: it either returns for an existing
enforcer or `Group.setBestPlan` retains the cheaper scan under that same key.
`chooseBestPlan` consequently selects both scans, and
`visitPhysicalNestedLoopJoin` merges them into one fragment.
`UnassignedJobBuilder.buildScanOlapTableJob` rejects that fragment with its
unsupported-multiple-OLAP-scans error, even on one BE. Forced execution gathers
need an identity/selection that cannot resolve back to StorageGather; repeating
the same helper call is insufficient. Add full/right outer non-equality NLJ
coverage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
