in which version of Hadoop you are seeing this?
-Vinay
On 29 Jan 2018 3:26 pm, "Xie Gang" <xiegang...@gmail.com> wrote:
Hello,
We recently hit a issue that almost all the disk of the datanode got full
even we configured the du .reserved.
After tracking down the code, found that when we choose a target datanode
and check if it's good candidate for block allocation (isGoodTarget()), it
only checks if the total left space of all the volumes(the same type), not
each volume. This logic makes the reservation of each volume useless.
Is this a problem or do I have any misunderstanding?
final long remaining = node.getRemaining(storage.getStorageType());
if (requiredSize > remaining - scheduledSize) {
logNodeIsNotChosen(storage, "the node does not have enough "
+ storage.getStorageType() + " space"
+ " (required=" + requiredSize
+ ", scheduled=" + scheduledSize
+ ", remaining=" + remaining + ")");
stats.incrOverScheduled();
return false;
}
--
Xie Gang
