ChenSammi commented on PR #4583: URL: https://github.com/apache/ozone/pull/4583#issuecomment-1527141829
> > From user's point of view, "hdds.datanode.volume.min.free.space" looks very similar to "hdds.datanode.dir.du.reserved" functionally, like another kind of reserved space. > > I just gave this a thought and realised that we had introduced this property because the flow is such that > > 1. Datanode runs DU/DF periodically and stores the disk stats to a cache , so at any point of time the stats might be behind the actual usage,( although this cache is updated on every write chunk so block data would be accounted in the space but it won't account for rocksdb/raft log metadata) > > 2. If we only configure a default for the reserved space, containers will close when usage crosses reserved space. There might be a delay in SCM receiving the close container action and asking the DN's in the pipeline to close it, during this delay client may still write to that container and violate the reserve space > > > If we are okay with crossing reserved space by a little then we can set a default for reserved space, else we need to have a small buffer like what is defined in this PR , before the reserved space is reached. Had a offline discussion with @sadanand48 , here are the agreed points, 1. This patch can help to reduce the possibility of run out of disk issue on DN. 2. Container Quota management(lease) will help to prevent DN from out of disk from SCM side. @sumitagrawl . 3. Besides the container block data, there are RocksDB directory and Ratis directory which are out of track about their storage usage. 4. Metrics of RocksDB directory usage and Ratis directory usage will help to estimate how much space should be reserved for them to avoid the DN out of space. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
