steveloughran commented on issue #1208: HADOOP-16423. S3Guard fsck: Check metadata consistency between S3 and metadatastore (log) URL: https://github.com/apache/hadoop/pull/1208#issuecomment-528866219 e.g ``` bin/hadoop s3guard fsck -check s3a://hwdev-steve-ireland-new/etc/hadoop/shellprofile.d 2019-09-06 14:51:46,377 [main] INFO s3guard.S3GuardTool (S3GuardTool.java:initMetadataStore(322)) - Metadata store DynamoDBMetadataStore{region=eu-west-1, tableName=hwdev-steve-ireland-new, tableArn=arn:aws:dynamodb:eu-west-1:980678866538:table/hwdev-steve-ireland-new} is initialized. 2019-09-06 14:51:46,705 [main] INFO s3guard.S3GuardFsck (S3GuardFsck.java:compareFileStatusToPathMetadata(217)) - Path: s3a://hwdev-steve-ireland-new/etc/hadoop/shellprofile.d - Length S3: 0, MS: 0 - Etag S3: null, MS: null 2019-09-06 14:51:46,764 [main] INFO s3guard.S3GuardFsck (S3GuardFsck.java:compareFileStatusToPathMetadata(217)) - Path: s3a://hwdev-steve-ireland-new/etc/hadoop/shellprofile.d/example.sh - Length S3: 3880, MS: 3880 - Etag S3: c7dbe1b877a287175df9dfc32c226765, MS: c7dbe1b877a287175df9dfc32c226765 2019-09-06 14:51:46,792 [main] ERROR s3guard.S3GuardFsckViolationHandler (S3GuardFsckViolationHandler.java:handle(79)) - On path: s3a://hwdev-steve-ireland-new/etc/hadoop/shellprofile.d No etag. 2019-09-06 14:51:46,792 [main] ERROR s3guard.S3GuardFsckViolationHandler (S3GuardFsckViolationHandler.java:handle(79)) - On path: s3a://hwdev-steve-ireland-new/etc/hadoop/shellprofile.d/example.sh getModificationTime mismatch - s3: 1567773091000, ms: 1567773090841 2019-09-06 14:51:46,792 [main] INFO s3guard.S3GuardFsck (S3GuardFsck.java:compareS3ToMs(144)) - Total scan time: 0s 2019-09-06 14:51:46,792 [main] INFO s3guard.S3GuardFsck (S3GuardFsck.java:compareS3ToMs(145)) - Scanned entries: 2 ~/P/R/fsck echo $status 0 ``` The good news: the return code is 0; it passed the scan. So these are just info rather than warn
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
