ulysses-you commented on a change in pull request #32932:
URL: https://github.com/apache/spark/pull/32932#discussion_r656693575
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala
##########
@@ -1351,6 +1351,26 @@ object RepartitionByExpression {
}
}
+/**
+ * This operator does not guarantee the output partitioning, because the
partition number will be
+ * optimized by AQE.
+ */
+case class AdaptiveRepartition(
+ partitionExpressions: Seq[Expression],
+ child: LogicalPlan) extends RepartitionOperation {
Review comment:
yea it's more safer to don't extend `RepartitionOperation`.
##########
File path: docs/sql-performance-tuning.md
##########
@@ -228,6 +228,8 @@ The "REPARTITION_BY_RANGE" hint must have column names and
a partition number is
SELECT /*+ REPARTITION */ * FROM t
SELECT /*+ REPARTITION_BY_RANGE(c) */ * FROM t
SELECT /*+ REPARTITION_BY_RANGE(3, c) */ * FROM t
+ SELECT /*+ ADAPTIVE_REPARTITION */ * FROM t
+ SELECT /*+ ADAPTIVE_REPARTITION(c) */ * FROM t
Review comment:
+1 for `REPARTITION_BY_AQE `
##########
File path: docs/sql-performance-tuning.md
##########
@@ -228,6 +228,8 @@ The "REPARTITION_BY_RANGE" hint must have column names and
a partition number is
SELECT /*+ REPARTITION */ * FROM t
SELECT /*+ REPARTITION_BY_RANGE(c) */ * FROM t
SELECT /*+ REPARTITION_BY_RANGE(3, c) */ * FROM t
+ SELECT /*+ ADAPTIVE_REPARTITION */ * FROM t
+ SELECT /*+ ADAPTIVE_REPARTITION(c) */ * FROM t
For more details please refer to the documentation of [Partitioning
Hints](sql-ref-syntax-qry-select-hints.html#partitioning-hints).
Review comment:
yea, it's better to add docs for the new hint. Since this PR only adds a
new hint that has no big difference with previous hint, we can add more docs
and use case after adding new rule.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
