Siyao Meng created HDDS-13569:
---------------------------------
Summary: Improve metadata dirs separation for OM and SCM in
MiniOzoneClusterImpl
Key: HDDS-13569
URL: https://issues.apache.org/jira/browse/HDDS-13569
Project: Apache Ozone
Issue Type: Task
Components: test
Reporter: Siyao Meng
Assignee: Siyao Meng
Currently MiniOzoneClusterImpl (non-HA) mixes OM and SCM metadata dirs all
under {{ozone-meta}}:
{code:title=Dir tree under
./hadoop-ozone/integration-test/target/test-dir/MiniOzoneClusterImpl-53868f3f-ffb9-495f-96ab-da3f551b9f06/}
.
└── ozone-meta
├── datanode-1
│ ├── data-0
│ │ └── hdds
│ │ └── 53868f3f-ffb9-495f-96ab-da3f551b9f06
│ │ ├── current
│ │ │ └── containerDir0
│ │ │ ├── 1
│ │ │ │ ├── chunks
│ │ │ │ └── metadata
│ │ │ ├── 2
│ │ │ │ ├── chunks
│ │ │ │ └── metadata
│ │ ├── DS-974ed241-60c2-416c-aff0-4074acd5165f
│ │ │ ├── container.db
│ │ │ ├── db.checkpoints
│ │ │ └── db.snapshots
│ │ │ └── checkpointState
│ │ └── tmp
│ │ ├── deleted-containers
│ │ └── disk-check
│ ├── meta
│ │ ├── db.checkpoints
│ │ ├── db.snapshots
│ │ │ └── checkpointState
│ │ ├── dnlayoutversion
│ │ ├── webserver
│ │ └── witnessed_container.db
│ └── ratis
│ ├── 15d8f352-a9ca-474c-afe5-6a154bb0694f
│ │ ├── current
│ │ └── sm
│ ├── 82cd09bf-ad27-44c0-b979-5625bf6ad9f4
│ │ ├── current
│ │ └── sm
│ ├── ac9d6dfc-16c0-4796-95a1-7b50c331b1e2
│ │ ├── current
│ │ └── sm
│ └── tmp
│ └── disk-check
├── datanode-2
...
├── datanode-3
...
├── db.checkpoints
├── db.snapdiff
├── db.snapshots
│ ├── checkpointState
│ └── diffState
│ ├── compaction-log
│ ├── compaction-sst-backup
│ └── snapDiff
├── om
│ └── current
├── om.db
│ └── archive
├── ratis
│ └── bf265839-605b-3f16-9796-c5ba1605619e
│ ├── current
│ └── sm
├── scm
│ └── current
├── scm-ha
│ └── 53868f3f-ffb9-495f-96ab-da3f551b9f06
│ ├── current
│ └── sm
├── scm.db
│ └── archive
├── snapshot
└── webserver
{code}
This makes it hard to calculate OM metadata real usage.
Also, some dirs are incorrectly shared among OM and SCM, such as {{snapshot}}
and {{webserver}}.
The goal is to separate those dirs for OM and SCM like we would do in prod.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
