clintropolis commented on a change in pull request #9935:
URL: https://github.com/apache/druid/pull/9935#discussion_r446466165
##########
File path:
server/src/main/java/org/apache/druid/metadata/IndexerSQLMetadataStorageCoordinator.java
##########
@@ -932,83 +930,100 @@ public int deletePendingSegments(String dataSource)
* Attempts to insert a single segment to the database. If the segment
already exists, will do nothing; although,
* this checking is imperfect and callers must be prepared to retry their
entire transaction on exceptions.
*
- * @return true if the segment was added, false if it already existed
+ * @return DataSegment set inserted
*/
- private boolean announceHistoricalSegment(
+ private Set<DataSegment> announceHistoricalSegmentBatch(
final Handle handle,
- final DataSegment segment,
- final boolean used
+ final Set<DataSegment> segments,
+ final Set<DataSegment> usedSegments
) throws IOException
{
+ final Set<DataSegment> toInsertSegments = new HashSet<>();
try {
- if (segmentExists(handle, segment)) {
- log.info("Found [%s] in DB, not updating DB", segment.getId());
- return false;
+ Set<String> existedSegments = segmentExistsBatch(handle, segments);
+ for (DataSegment segment : segments) {
+ if (existedSegments.contains(segment.getId().toString())) {
+ log.info("Found [%s] in DB, not updating DB", segment.getId());
Review comment:
There is a
[`log.infoSegments`](https://github.com/apache/druid/blob/master/core/src/main/java/org/apache/druid/java/util/common/logger/Logger.java#L182)
method that takes a collection of segments that should be used if logging a
potentially large list of segments.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
