gianm commented on a change in pull request #7595: Optimize overshadowed
segments computation
URL: https://github.com/apache/incubator-druid/pull/7595#discussion_r288010461
##########
File path:
server/src/main/java/org/apache/druid/metadata/SQLMetadataSegmentManager.java
##########
@@ -744,6 +755,35 @@ public DataSegment map(int index, ResultSet r,
StatementContext ctx) throws SQLE
// Replace "dataSources" atomically.
Review comment:
"dataSources" and "overshadowedSegments" should be updated atomically,
together. Updating them separately means we could get info about overshadowed
segments without actually knowing they're overshadowed (or vice versa). The
simplest way to do this is to wrap them into a single object and then swap in
_that_ object atomically.
One way to do this is to make `getDataSources()` return
`Collection<DataSourceSnapshot>` instead of
`Collection<ImmutableDruidDataSource>`, and make `DataSourceSnapshot` something
that contains both an `ImmutableDruidDataSource` and an
`ImmutableSet<SegmentId> overshadowedSegments`. For each one, build the
overshadowedSegments list by running
`VersionedIntervalTimeline.forSegments(...)` on the segments for that
datasource, and then extracting the overshadowed ones.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
