abhishekagarwal87 commented on a change in pull request #10884:
URL: https://github.com/apache/druid/pull/10884#discussion_r578269049
##########
File path:
server/src/main/java/org/apache/druid/segment/loading/StorageLocation.java
##########
@@ -117,6 +117,33 @@ public synchronized File reserve(String segmentDir,
DataSegment segment)
return reserve(segmentDir, segment.getId().toString(), segment.getSize());
}
+ /**
+ * Reserves space to store the given segment, only if it has not been done
already. This can be used
+ * when segment is already downloaded on the disk. Unlike {@link
#reserve(String, DataSegment)}, this function
+ * skips the check on disk availability. We also account for segment usage
even if available size dips below 0.
+ * Such a situation indicates a configuration problem or a bug and we don't
let segment loading fail because
+ * of this.
+ */
+ public synchronized void maybeReserve(String segmentFilePathToAdd,
DataSegment segment)
+ {
+ final File segmentFileToAdd = new File(path, segmentFilePathToAdd);
+ if (files.contains(segmentFileToAdd)) {
+ // Already reserved
+ return;
+ }
+ files.add(segmentFileToAdd);
+ currSizeBytes += segment.getSize();
+ if (availableSizeBytes() < 0) {
+ log.error(
Review comment:
> It might be worth upgrading the log level to alert for this, since I
think it'd be a good idea to take prompt action when this happens.
Makes sense.
> What kind of configuration problems do you foresee leading to this error
(would it happen if a user drops the maxSize on a historical with a filled
segment cache? are there other cases?)
I don't expect this to happen unless someone reduced the max size or you
re-attached a different data disk to the node. Most likely, if it happens, it's
a bug somewhere in the code.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
