JacksonYao287 edited a comment on pull request #2182:
URL: https://github.com/apache/ozone/pull/2182#issuecomment-843886787
@bshashikant , thanks for the review!
for now, there are two kinds of report: IncrementalReport, which will be put
into incrementalMessageQueue, and FullReport, which will be refreshed
periodically.
for IncrementalReport, if container/pipeline take new actions, they want
SCM to know this action is done as soon as possible by sending through sendICR
and triggering a new heartbeat proactively , so that SCM can be acquired this
and update the corresponding state of container/pipeline as fast as possible
and then take the next action.(eg, reply to OM).
for FullReport , they are published periodically and controlled by
`hdds.node.report.interval`,`hdds.container.report.interval`,`hdds.pipeline.report.interval`,
so if these parameters are longer than heartbeat interval, they should not
exist in every heartbeat information.
>A report ideally means a full report status of datanode to SCM
so I think a report may not always mean a full report. In my test
environment , there are about a million containers. if scm is always acquired
full container report, i think it will be a heavy burden for network, scm and
datanode
for now , there are two problems.
1 acorrding to the current implementation, heartbeat will always report full
report , even if they just only want to send an incremental report,and thus
`hdds.node.report.interval`,`hdds.container.report.interval`,`hdds.pipeline.report.interval`
do not take effect , i think this is a bug , and i fix it.
2 current logic of "scmcontex#addreport" is a little confused, so i refactor
this , split it into to two functions, so that the logic is now clear.
```
private void sendPipelineReport() {
if (context != null) {
// TODO: Send IncrementalPipelineReport instead of full PipelineReport
context.addReport(context.getParent().getContainer().getPipelineReport());
context.getParent().triggerHeartbeat();
}
}
```
after refactoring , now we can send IncrementalPipelineReport instead of
full PipelineReport
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
