timmylicheng edited a comment on issue #393: HDDS-1590 Aggregate bytes write metrics from container to datanode and to SCM URL: https://github.com/apache/hadoop-ozone/pull/393#issuecomment-588299820 > > Sure. This patch is to summarize the bytes in for every pipeline, which requires the sum of each container that is allocated to the pipeline. The bytes in stats for each container shall be collected on datanode to be near real-time. So that I collect them up by datanode reports and aggregate by pipeline on SCM. > > Thanks to explain it. > > I understand that it's good to calculate the number. I don't understand why is it added to the `StorageContainerDatanodeProtocol.proto` and sent back to the SCM. It seems to be a metric which can be fetched directly from the datanode. We don't need to send it to the SCM. > > Unless the SCM needs this data for any **decision** (to create new pipelines, replica containers, etc..). I might missed it but as I see this is not the case (but please let me now if you have such a plan...). Yea I think you already have the picture. We plan to expose pipeline heavy load in terms of bytes written to make pipeline lifecycle more balanced. @elek
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
