ankitsultana commented on code in PR #15813:
URL: https://github.com/apache/pinot/pull/15813#discussion_r2093740781
##########
pinot-query-planner/src/main/java/org/apache/pinot/query/planner/physical/v2/ExchangeStrategy.java:
##########
@@ -18,50 +18,100 @@
*/
package org.apache.pinot.query.planner.physical.v2;
+import java.util.List;
+import org.apache.calcite.rel.RelDistribution;
+import org.apache.calcite.rel.RelDistributions;
+
+
/**
* Defines how data is transferred across an Exchange.
*/
public enum ExchangeStrategy {
/**
* There's a single stream in the receiver, so each stream in the sender
sends data to the same.
*/
- SINGLETON_EXCHANGE,
+ SINGLETON_EXCHANGE(false),
/**
* stream-ID X sends data to stream-ID X. This cannot be modeled by
PARTITIONING_EXCHANGE because the fan-out for
* this type of exchange is 1:1.
*/
- IDENTITY_EXCHANGE,
+ IDENTITY_EXCHANGE(false),
/**
* Each stream will partition the outgoing stream based on a set of keys and
a hash function.
* Fanout for this type of exchange is 1:all.
*/
- PARTITIONING_EXCHANGE,
+ PARTITIONING_EXCHANGE(true),
/**
* stream-ID X will sub-partition: i.e. divide the stream so that the data
is sent to the streams
* {@code X, X + F, X + 2*F, ...}. Here F is the sub-partitioning factor.
Records are assigned based on a
* hash function. This is useful when joining two tables which have
different number of partitions, but one of the
* partition counts divides the other.
* <b>Note:</b> This is different and better than partitioning exchange
because the fanout is F, and not N * (N*F).
*/
- SUB_PARTITIONING_HASH_EXCHANGE,
+ SUB_PARTITIONING_HASH_EXCHANGE(true),
/**
* Same as above but records are sub-partitioned in a round-robin way. This
will increase parallelism but lose
* data partitioning.
*/
- SUB_PARTITIONING_RR_EXCHANGE,
+ SUB_PARTITIONING_RR_EXCHANGE(false),
Review Comment:
This can be helpful when you have a scenario where parallelism changes but
there's no hash partitioning to maintain. (e.g. you go from 4 workers to 8
workers, but the data was un-partitioned)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
