kerneltime commented on code in PR #7127:
URL: https://github.com/apache/ozone/pull/7127#discussion_r1772323593
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/BackgroundContainerDataScanner.java:
##########
@@ -87,27 +87,30 @@ public void scanContainer(Container<?> c)
ContainerData containerData = c.getContainerData();
long containerId = containerData.getContainerID();
logScanStart(containerData);
- ScanResult result = c.scanData(throttler, canceler);
-
- // Metrics for skipped containers should not be updated.
- if (result.getFailureType() == DELETED_CONTAINER) {
- LOG.error("Container [{}] has been deleted.",
- containerId, result.getException());
- return;
- }
- if (!result.isHealthy()) {
- LOG.error("Corruption detected in container [{}]. Marking it UNHEALTHY.",
- containerId, result.getException());
- boolean containerMarkedUnhealthy =
controller.markContainerUnhealthy(containerId, result);
- if (containerMarkedUnhealthy) {
- metrics.incNumUnHealthyContainers();
+ DataScanResult result = c.scanData(throttler, canceler);
Review Comment:
Nit: Not part of this PR but the whole `scanContainer` should have a metric
around time taken
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/BackgroundContainerDataScanner.java:
##########
@@ -87,27 +87,30 @@ public void scanContainer(Container<?> c)
ContainerData containerData = c.getContainerData();
long containerId = containerData.getContainerID();
logScanStart(containerData);
- ScanResult result = c.scanData(throttler, canceler);
-
- // Metrics for skipped containers should not be updated.
- if (result.getFailureType() == DELETED_CONTAINER) {
- LOG.error("Container [{}] has been deleted.",
- containerId, result.getException());
- return;
- }
- if (!result.isHealthy()) {
- LOG.error("Corruption detected in container [{}]. Marking it UNHEALTHY.",
- containerId, result.getException());
- boolean containerMarkedUnhealthy =
controller.markContainerUnhealthy(containerId, result);
- if (containerMarkedUnhealthy) {
- metrics.incNumUnHealthyContainers();
+ DataScanResult result = c.scanData(throttler, canceler);
Review Comment:
NIT:
```
if (volume.isFailed()) {
shutdown("The volume has failed.");
return;
}
```
We should log volume info in the log message in line 80.
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/BackgroundContainerDataScanner.java:
##########
@@ -87,27 +87,30 @@ public void scanContainer(Container<?> c)
ContainerData containerData = c.getContainerData();
long containerId = containerData.getContainerID();
logScanStart(containerData);
- ScanResult result = c.scanData(throttler, canceler);
-
- // Metrics for skipped containers should not be updated.
- if (result.getFailureType() == DELETED_CONTAINER) {
- LOG.error("Container [{}] has been deleted.",
- containerId, result.getException());
- return;
- }
- if (!result.isHealthy()) {
- LOG.error("Corruption detected in container [{}]. Marking it UNHEALTHY.",
- containerId, result.getException());
- boolean containerMarkedUnhealthy =
controller.markContainerUnhealthy(containerId, result);
- if (containerMarkedUnhealthy) {
- metrics.incNumUnHealthyContainers();
+ DataScanResult result = c.scanData(throttler, canceler);
+
+ if (result.isDeleted()) {
+ LOG.debug("Container [{}] has been deleted during the data scan.",
containerId);
+ } else {
+ if (!result.isHealthy()) {
+ logUnhealthyScanResult(containerId, result, LOG);
+
+ // Only increment the number of unhealthy containers if the container
was not already unhealthy.
+ if (controller.markContainerUnhealthy(containerId, result)) {
+ metrics.incNumUnHealthyContainers();
Review Comment:
This method does not check if before the scan the container was healthy,
then before incrementing the metric we should check if there is a state
transition or if the container was already unhealthy.
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/ContainerData.java:
##########
@@ -78,7 +78,7 @@ public abstract class ContainerData {
private String chunksPath;
// State of the Container
- private ContainerDataProto.State state;
Review Comment:
All the getters and setters on this are `synchronized`, why do we need
`volatile`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
