shauryachats commented on code in PR #18467:
URL: https://github.com/apache/pinot/pull/18467#discussion_r3263662831
##########
pinot-server/src/main/java/org/apache/pinot/server/predownload/PredownloadSegmentInfo.java:
##########
@@ -172,9 +176,27 @@ public boolean hasSameCRC() {
}
}
- public void updateSegmentInfoFromLocal(@Nullable SegmentDirectory
segmentDirectory) {
- SegmentMetadataImpl segmentMetadata = (segmentDirectory == null) ? null :
segmentDirectory.getSegmentMetadata();
- _localCrc = (segmentMetadata == null) ? null : segmentMetadata.getCrc();
- _localSizeBytes = (segmentDirectory == null) ? 0 :
segmentDirectory.getDiskSizeBytes();
+ /**
+ * Populates local CRC and size from the segment directory on disk, avoiding
mmap of index files.
+ * Reads CRC from {@code creation.meta} (8 bytes) and computes size via
directory traversal.
+ * Sets {@code _localCrc} to null and {@code _localSizeBytes} to 0 if the
segment or its
+ * creation.meta does not exist.
+ */
+ public void updateSegmentInfoFromLocal(File segDir) {
+ if (!segDir.isDirectory()) {
+ LOGGER.warn("Segment path is not a directory: {}", segDir);
+ return;
+ }
+ File creationMeta = SegmentDirectoryPaths.findCreationMetaFile(segDir);
+ if (creationMeta == null || !creationMeta.exists()) {
+ return;
+ }
+ try (DataInputStream ds = new DataInputStream(new
FileInputStream(creationMeta))) {
+ _localCrc = String.valueOf(ds.readLong());
+ } catch (IOException e) {
+ LOGGER.warn("Failed to read creation.meta for segment: {} of table: {}",
_segmentName, _tableNameWithType, e);
+ return;
+ }
+ _localSizeBytes = FileUtils.sizeOfDirectory(segDir);
Review Comment:
Let's wrap this within a try-catch with `IllegalArgumentException`: a TOCTOU
race could happen between the `!segDir.isDirectory()` check and the
`FileUtils.sizeOfDirectory` if the directory itself is deleted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
