xiangqiao123 commented on a change in pull request #9935:
URL: https://github.com/apache/druid/pull/9935#discussion_r449577265
##########
File path:
server/src/main/java/org/apache/druid/metadata/IndexerSQLMetadataStorageCoordinator.java
##########
@@ -932,83 +930,100 @@ public int deletePendingSegments(String dataSource)
* Attempts to insert a single segment to the database. If the segment
already exists, will do nothing; although,
* this checking is imperfect and callers must be prepared to retry their
entire transaction on exceptions.
*
- * @return true if the segment was added, false if it already existed
+ * @return DataSegment set inserted
*/
- private boolean announceHistoricalSegment(
+ private Set<DataSegment> announceHistoricalSegmentBatch(
final Handle handle,
- final DataSegment segment,
- final boolean used
+ final Set<DataSegment> segments,
+ final Set<DataSegment> usedSegments
) throws IOException
{
+ final Set<DataSegment> toInsertSegments = new HashSet<>();
try {
- if (segmentExists(handle, segment)) {
- log.info("Found [%s] in DB, not updating DB", segment.getId());
- return false;
+ Set<String> existedSegments = segmentExistsBatch(handle, segments);
+ for (DataSegment segment : segments) {
+ if (existedSegments.contains(segment.getId().toString())) {
+ log.info("Found [%s] in DB, not updating DB", segment.getId());
Review comment:
> This will corrupt the overlord logs by printing the similar logs over
again. How about printing them all at once? You can do by doing like this:
>
> ```java
> Set<String> existedSegments = segmentExistsBatch(handle, segments);
> log.info("Found these segments already exist in DB: %s",
existedSegments);
> for (DataSegment segment : segments) {
> if (!existedSegments.contains(segment.getId().toString())) {
> toInsertSegments.add(segment);
> }
> }
> ```
Modified as required.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
