clintropolis commented on a change in pull request #8281: Add realization for
updating version of derived segments in Materiali…
URL: https://github.com/apache/incubator-druid/pull/8281#discussion_r317022789
##########
File path:
extensions-contrib/materialized-view-maintenance/src/main/java/org/apache/druid/indexing/materializedview/MaterializedViewSupervisor.java
##########
@@ -369,6 +369,19 @@ void checkSegmentsAndSubmitTasks()
MapDifference<Interval, String> difference =
Maps.difference(maxCreatedDate, derivativeVersion);
Map<Interval, String> toBuildInterval = new
HashMap<>(difference.entriesOnlyOnLeft());
Map<Interval, String> toDropInterval = new
HashMap<>(difference.entriesOnlyOnRight());
+ // update version of derived segments if isn't the max (created_date) of
all base segments
+ // prevent user supplied segments list did not match with segments list
obtained from db
+ Map<Interval, MapDifference.ValueDifference<String>> checkIfNewestVersion =
+ new HashMap<>(difference.entriesDiffering());
+ for (Map.Entry<Interval, MapDifference.ValueDifference<String>> entry :
checkIfNewestVersion.entrySet()) {
+ String versionOfBase = maxCreatedDate.get(entry.getKey());
+ String versionOfDerivative = derivativeVersion.get(entry.getKey());
+ if (versionOfBase.compareTo(versionOfDerivative) > 0 &&
Review comment:
While I'm still trying to figure out the details on this extension, I do I
think for the sake of ease of understanding what is going on here that maybe
all of the values compared in this `if` statement should be pulled out as
variables instead of just the versions, maybe something like:
```
final String versionOfBase = maxCreatedDate.get(entry.getKey());
final String versionOfDerivative =
derivativeVersion.get(entry.getKey());
final int baseCount = baseSegments.get(entry.getKey()).size();
final IntSupplier usedCountSupplier =
() ->
metadataStorageCoordinator.getUsedSegmentsForInterval(spec.getBaseDataSource(),
entry.getKey()).size();
if (versionOfBase.compareTo(versionOfDerivative) > 0 && baseCount ==
usedCountSupplier.getAsInt()) {
toBuildInterval.put(entry.getKey(), versionOfBase);
}
```
However, up to you if you want to change this or not.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
