abhishekrb19 commented on code in PR #15243:
URL: https://github.com/apache/druid/pull/15243#discussion_r1381070845
##########
indexing-service/src/main/java/org/apache/druid/indexing/common/task/batch/parallel/TombstoneHelper.java:
##########
@@ -173,66 +190,69 @@ public Set<DataSegment>
computeTombstoneSegmentsForReplace(
* They should be aligned with the
replaceGranularity
* @param dataSource Datasource on which the replace is to be
performed
* @param replaceGranularity Granularity of the replace query
+ * @param maxBuckets Maximum number of partition buckets. If the
number of computed tombstone buckets
+ * exceeds this threshold, the method will throw
an error.
* @return Intervals computed for the tombstones
* @throws IOException
*/
public Set<Interval> computeTombstoneIntervalsForReplace(
List<Interval> intervalsToDrop,
List<Interval> intervalsToReplace,
String dataSource,
- Granularity replaceGranularity
+ Granularity replaceGranularity,
+ int maxBuckets
) throws IOException
{
Set<Interval> retVal = new HashSet<>();
List<Interval> usedIntervals =
getExistingNonEmptyIntervalsOfDatasource(intervalsToReplace, dataSource);
+ int buckets = 0;
for (Interval intervalToDrop : intervalsToDrop) {
for (Interval usedInterval : usedIntervals) {
Interval overlap = intervalToDrop.overlap(usedInterval);
- // No overlap of the dropped segment with the used interval due to
which we donot need to generate any tombstone
+ // No overlap of the dropped segment with the used interval due to
which we do not need to generate any tombstone
if (overlap == null) {
continue;
}
+ if (buckets > maxBuckets) {
+ throw new IAE("Cannot add more tombstone buckets than [%d].",
maxBuckets);
Review Comment:
I considered returning the MSQ fault `TooManyBucketsFault`
~`TooManyBucketsException`~ directly here. But since this utility resides in
the common indexing package, we cannot use an MSQ fault here. So it has to be
a more generic exception or introduce other semantics.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
