gianm commented on a change in pull request #6155: Fix three bugs with segment publishing. URL: https://github.com/apache/incubator-druid/pull/6155#discussion_r210139522
########## File path: server/src/main/java/io/druid/segment/realtime/appenderator/BaseAppenderatorDriver.java ########## @@ -555,38 +555,33 @@ protected AppenderatorDriverAddResult append( final boolean published = publisher.publishSegments( ImmutableSet.copyOf(segmentsAndMetadata.getSegments()), metadata == null ? null : ((AppenderatorDriverMetadata) metadata).getCallerMetadata() - ); + ).isSuccess(); if (published) { log.info("Published segments."); } else { - log.info("Transaction failure while publishing segments, checking if someone else beat us to it."); + log.info("Transaction failure while publishing segments, removing them from deep storage " + + "and checking if someone else beat us to publishing."); + + segmentsAndMetadata.getSegments().forEach(dataSegmentKiller::killQuietly); + final Set<SegmentIdentifier> segmentsIdentifiers = segmentsAndMetadata .getSegments() .stream() .map(SegmentIdentifier::fromDataSegment) .collect(Collectors.toSet()); + if (usedSegmentChecker.findUsedSegments(segmentsIdentifiers) .equals(Sets.newHashSet(segmentsAndMetadata.getSegments()))) { - log.info( - "Removing our segments from deep storage because someone else already published them: %s", - segmentsAndMetadata.getSegments() - ); - segmentsAndMetadata.getSegments().forEach(dataSegmentKiller::killQuietly); - log.info("Our segments really do exist, awaiting handoff."); } else { - throw new ISE("Failed to publish segments[%s]", segmentsAndMetadata.getSegments()); + throw new ISE("Failed to publish segments."); Review comment: I was thinking it's not necessary, since they will get logged in the catch statement via: ```java log.warn(e, "Failed publish, not removing segments: %s", segmentsAndMetadata.getSegments()); ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org