[
https://issues.apache.org/jira/browse/HDDS-6598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Gui updated HDDS-6598:
---------------------------
Description:
After stressing a cluster for several days, we found that there are a lot of
CLOSED EC pipelines.
{code:java}
[ozoneadmin@TENCENT64 ~/ozone-1.3.0-SNAPSHOT]$ ./bin/ozone admin pipeline list
--state=CLOSED | wc -l
997 {code}
It makes commands return slowly(e.g. ozone admin datanode list, ozone admin
pipeline list), and potentially it will add unnecessary burden to SCM HA, so
these CLOSED EC pipelines should be cleaned up properly.
Several ways to consider:
- We close pipelines in `WritableECContainerProvider` by calling
`pipelineManager.closePipeline(pipeline, true);`
was:
After stressing a cluster for several days, we found that EC pipeline records
are never removed:
[ozoneadmin@TENCENT64 ~/ozone-1.3.0-SNAPSHOT]$ ./bin/ozone admin pipeline list
--state=CLOSED | wc -l
997
> EC: EC pipeline records are not removed after close.
> ----------------------------------------------------
>
> Key: HDDS-6598
> URL: https://issues.apache.org/jira/browse/HDDS-6598
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Mark Gui
> Assignee: Mark Gui
> Priority: Major
>
> After stressing a cluster for several days, we found that there are a lot of
> CLOSED EC pipelines.
> {code:java}
> [ozoneadmin@TENCENT64 ~/ozone-1.3.0-SNAPSHOT]$ ./bin/ozone admin pipeline
> list --state=CLOSED | wc -l
> 997 {code}
> It makes commands return slowly(e.g. ozone admin datanode list, ozone admin
> pipeline list), and potentially it will add unnecessary burden to SCM HA, so
> these CLOSED EC pipelines should be cleaned up properly.
> Several ways to consider:
> - We close pipelines in `WritableECContainerProvider` by calling
> `pipelineManager.closePipeline(pipeline, true);`
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
