errose28 commented on PR #8755: URL: https://github.com/apache/ozone/pull/8755#issuecomment-3099100461
> Are you telling that we should not rely on Recon for capacity distribution? "Capacity Distribution" will need a tighter definition in order to make this call. We should not rely on Recon for time series data, and the information added in this PR is time series data. > Alternatively, we could introduce a new endpoint at the DataNode level, which Recon would call to gather and aggregate this information. You have just described the process of Prometheus's metrics scraping : ) This is why we should not use Recon for this since Prometheus already handles this. > since the storage report already includes other capacity-related fields, we considered extending it to include pending deletion information as well — which would simplify the implementation. I don't see any evidence supporting this simplification claim. Making Ozone do this is far more complex to implement than leaving it to existing systems designed to handle time series data. > Given that this data ultimately needs to be available at the Recon level — especially to enhance the current high-level view with more granular insights (e.g., how much space can be reclaimed after deletions or where deletions are stuck) I disagree that this needs to be available in Recon. Recon is not designed to store or display time-series data. The terminology "stuck" used here is an indication that we are dealing with times series data, because it translates to deletion progress over time. > which approach do you think is better? Publish this time series data as metrics from the datanodes and let existing systems handle it much better than we can. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
