[
https://issues.apache.org/jira/browse/HDDS-17772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Yarovoy updated HDDS-17772:
----------------------------------
Description:
h3. Problem
HDDS-15342 (a3ca93ac43, "Make RocksDB bottommost level compaction options
configurable for background compaction") added an OM setting to
{{{}OMConfigKeys{}}}, in {{{}ozone-common{}}}, whose default is a RocksDB type:
{code:java}
import org.rocksdb.CompactRangeOptions.BottommostLevelCompaction;
...
public static final BottommostLevelCompaction
OZONE_OM_COMPACTION_SERVICE_BOTTOMMOSTLEVELCOMPACTION_DEFAULT =
BottommostLevelCompaction.kSkip;
{code}
To compile it, the change added {{org.rocksdb:rocksdbjni}} to
{{hadoop-ozone/common/pom.xml}} at compile scope.
{{ozone-common}} is a client dependency, so every client artifact built on it
now carries RocksDB:
{noformat}
org.apache.ozone:ozone-filesystem-shaded
\- org.apache.ozone:ozone-filesystem-common
\- org.apache.ozone:ozone-common
\- org.rocksdb:rocksdbjni:10.10.1.1:compile
{noformat}
||version||jar size||RocksDB content||
|2.2.0|68.8 MB|none|
|2.3.0-SNAPSHOT (master)|155.0 MB|253 shaded {{org.rocksdb}} classes, and
{{librocksdbjni-*}} native libraries for 15 platforms (Linux x86, ARM, PowerPC,
s390x, RISC-V and 32-bit, glibc and musl; macOS; Windows): 83.6 MB compressed,
about 230 MB unpacked|
Nearly all of the growth, 83.6 of 86.2 MB, is RocksDB. No client code uses it:
that import is the only use of {{org.rocksdb}} in {{{}ozone-common{}}}.
HDDS-15466 had kept RocksDB out of {{ozone-common}} for the same option by
passing it as a plain number ({{{}OMAdminProtocol.compactOMDB(String, int){}}}).
h3. Impact
* Every client artifact and classpath that includes {{ozone-common}} is about
87 MB larger.
* {{ozone-filesystem-hadoop3}} is the jar that MapReduce and Spark jobs ship
with {{-libjars-}} or {{{}-jars{}}}, so every container that localizes it
downloads 155 MB instead of 68 MB. In a TestDFSIO comparison on a 12-node
cluster, after an upgrade to a build with this change, Ozone jobs' AppMaster
start grew by about 0.6 s and map tasks by 0.4–0.9 s,
while HDFS jobs on the same cluster didn't change. Whether the jar is the whole
cause is being measured by comparing shipped and preinstalled jars.
* Applications that bundle the Ozone client in fat jars or images grow by the
same amount.
h3. Proposed fix
* In {{{}OMConfigKeys{}}}, make the default a string, {{{}"kSkip"{}}}, with no
RocksDB import.
* Have {{CompactDBUtil.getBottommostLevelCompaction}} (ozone-manager, which
already depends on RocksDB) parse it into the enum, as it already does for
configured values.
* Remove {{rocksdbjni}} from {{{}hadoop-ozone/common/pom.xml{}}}.
* Adapt {{{}TestCompactDBUtil{}}}, which uses the default as an enum.
* Guard against it coming back, by either:
** an enforcer {{bannedDependencies}} rule for {{org.rocksdb:rocksdbjni}} in
the client modules ({{{}ozone-common{}}},
{{{}ozone-client{}}}, {{{}ozone-filesystem-*{}}}); or
** a check in {{ozonefs-shaded}} that the shaded jar has no {{org/rocksdb}}
classes or {{librocksdbjni}} files.
h3. How to check
mvn dependency:tree -pl :ozone-filesystem-shaded
-Dincludes=org.rocksdb:rocksdbjni # empty after the fix
unzip hadoop-ozone/ozonefs-hadoop3/target/ozone-filesystem-hadoop3*.jar | grep
-c librocksdbjni # 0
ls -- hadoop-ozone/ozonefs-hadoop3/target/ozone-filesystem-hadoop3*.jar #
about 70 MB
{noformat}
{noformat}
was:
h3. Problem
HDDS-15342 (a3ca93ac43, "Make RocksDB bottommost level compaction options
configurable for background compaction") added an OM setting to
\{{OMConfigKeys}}, in \{{ozone-common}}, whose default is a RocksDB type:
{code:java}
import org.rocksdb.CompactRangeOptions.BottommostLevelCompaction;
...
public static final BottommostLevelCompaction
OZONE_OM_COMPACTION_SERVICE_BOTTOMMOSTLEVELCOMPACTION_DEFAULT =
BottommostLevelCompaction.kSkip;
{code}
To compile it, the change added \{{org.rocksdb:rocksdbjni}} to
\{{hadoop-ozone/common/pom.xml}} at compile scope.
{\{ozone-common}} is a client dependency, so every client artifact built on it
now carries RocksDB:
{noformat}
org.apache.ozone:ozone-filesystem-shaded
\- org.apache.ozone:ozone-filesystem-common
\- org.apache.ozone:ozone-common
\- org.rocksdb:rocksdbjni:10.10.1.1:compile
{noformat}
||version||jar size||RocksDB content||
|2.2.0|68.8 MB|none|
|2.3.0-SNAPSHOT (master)|155.0 MB|253 shaded \{{org.rocksdb}} classes, and
\{{librocksdbjni-*}} native libraries for 15 platforms (Linux x86, ARM,
PowerPC, s390x, RISC-V and 32-bit, glibc and musl; macOS; Windows): 83.6 MB
compressed, about 230 MB unpacked|
Nearly all of the growth, 83.6 of 86.2 MB, is RocksDB. No client code uses it:
that import is the only use of \{{org.rocksdb}} in \{{ozone-common}}.
HDDS-15466 had kept RocksDB out of \{{ozone-common}} for the same option by
passing it as a plain number (\{{OMAdminProtocol.compactOMDB(String, int)}}).
h3. Impact
* Every client artifact and classpath that includes \{{ozone-common}} is about
87 MB larger.
* \{{ozone-filesystem-hadoop3}} is the jar that MapReduce and Spark jobs ship
with \{{-libjars}} or \{{--jars}}, so every container that localizes it
downloads 155 MB instead of 68 MB. In a TestDFSIO comparison on a 12-node
cluster, after an upgrade to a build with this change, Ozone jobs' AppMaster
start grew by about 0.6 s and map tasks by 0.4–0.9 s,
while HDFS jobs on the same cluster didn't change. Whether the jar is the whole
cause is being measured by comparing shipped and preinstalled jars.
* Applications that bundle the Ozone client in fat jars or images grow by the
same amount.
h3. Proposed fix
* In \{{OMConfigKeys}}, make the default a string, \{{"kSkip"}}, with no
RocksDB import.
* Have \{{CompactDBUtil.getBottommostLevelCompaction}} (ozone-manager, which
already depends on RocksDB) parse it into the enum, as it already does for
configured values.
* Remove \{{rocksdbjni}} from \{{hadoop-ozone/common/pom.xml}}.
* Adapt \{{TestCompactDBUtil}}, which uses the default as an enum.
* Guard against it coming back, by either:
** an enforcer \{{bannedDependencies}} rule for \{{org.rocksdb:rocksdbjni}} in
the client modules (\{{ozone-common}},
{\{ozone-client}}, \{{ozone-filesystem-*}}); or
** a check in \{{ozonefs-shaded}} that the shaded jar has no \{{org/rocksdb}}
classes or \{{librocksdbjni}} files.
h3. How to check
{noformat}
mvn dependency:tree -pl :ozone-filesystem-shaded
-Dincludes=org.rocksdb:rocksdbjni # empty after the fix
unzip -l hadoop-ozone/ozonefs-hadoop3/target/ozone-filesystem-hadoop3-*.jar |
grep -c librocksdbjni # 0
ls -l hadoop-ozone/ozonefs-hadoop3/target/ozone-filesystem-hadoop3-*.jar #
about 70 MB
> ozone-common depends on rocksdbjni since HDDS-15342, so client jars carry
> RocksDB: ozone-filesystem-hadoop3 grows from 68 MB to 155 MB
> --------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HDDS-17772
> URL: https://issues.apache.org/jira/browse/HDDS-17772
> Project: Apache Ozone
> Issue Type: Bug
> Components: common
> Affects Versions: 2.3.0
> Reporter: Andrey Yarovoy
> Assignee: Andrey Yarovoy
> Priority: Critical
>
> h3. Problem
> HDDS-15342 (a3ca93ac43, "Make RocksDB bottommost level compaction options
> configurable for background compaction") added an OM setting to
> {{{}OMConfigKeys{}}}, in {{{}ozone-common{}}}, whose default is a RocksDB
> type:
> {code:java}
> import org.rocksdb.CompactRangeOptions.BottommostLevelCompaction;
> ...
> public static final BottommostLevelCompaction
> OZONE_OM_COMPACTION_SERVICE_BOTTOMMOSTLEVELCOMPACTION_DEFAULT =
> BottommostLevelCompaction.kSkip;
> {code}
> To compile it, the change added {{org.rocksdb:rocksdbjni}} to
> {{hadoop-ozone/common/pom.xml}} at compile scope.
> {{ozone-common}} is a client dependency, so every client artifact built on it
> now carries RocksDB:
> {noformat}
> org.apache.ozone:ozone-filesystem-shaded
> \- org.apache.ozone:ozone-filesystem-common
> \- org.apache.ozone:ozone-common
> \- org.rocksdb:rocksdbjni:10.10.1.1:compile
> {noformat}
> ||version||jar size||RocksDB content||
> |2.2.0|68.8 MB|none|
> |2.3.0-SNAPSHOT (master)|155.0 MB|253 shaded {{org.rocksdb}} classes, and
> {{librocksdbjni-*}} native libraries for 15 platforms (Linux x86, ARM,
> PowerPC, s390x, RISC-V and 32-bit, glibc and musl; macOS; Windows): 83.6 MB
> compressed, about 230 MB unpacked|
> Nearly all of the growth, 83.6 of 86.2 MB, is RocksDB. No client code uses
> it: that import is the only use of {{org.rocksdb}} in {{{}ozone-common{}}}.
> HDDS-15466 had kept RocksDB out of {{ozone-common}} for the same option by
> passing it as a plain number ({{{}OMAdminProtocol.compactOMDB(String,
> int){}}}).
> h3. Impact
> * Every client artifact and classpath that includes {{ozone-common}} is
> about 87 MB larger.
> * {{ozone-filesystem-hadoop3}} is the jar that MapReduce and Spark jobs ship
> with {{-libjars-}} or {{{}-jars{}}}, so every container that localizes it
> downloads 155 MB instead of 68 MB. In a TestDFSIO comparison on a 12-node
> cluster, after an upgrade to a build with this change, Ozone jobs' AppMaster
> start grew by about 0.6 s and map tasks by 0.4–0.9 s,
> while HDFS jobs on the same cluster didn't change. Whether the jar is the
> whole cause is being measured by comparing shipped and preinstalled jars.
> * Applications that bundle the Ozone client in fat jars or images grow by
> the same amount.
> h3. Proposed fix
> * In {{{}OMConfigKeys{}}}, make the default a string, {{{}"kSkip"{}}}, with
> no RocksDB import.
> * Have {{CompactDBUtil.getBottommostLevelCompaction}} (ozone-manager, which
> already depends on RocksDB) parse it into the enum, as it already does for
> configured values.
> * Remove {{rocksdbjni}} from {{{}hadoop-ozone/common/pom.xml{}}}.
> * Adapt {{{}TestCompactDBUtil{}}}, which uses the default as an enum.
> * Guard against it coming back, by either:
> ** an enforcer {{bannedDependencies}} rule for {{org.rocksdb:rocksdbjni}} in
> the client modules ({{{}ozone-common{}}},
> {{{}ozone-client{}}}, {{{}ozone-filesystem-*{}}}); or
> ** a check in {{ozonefs-shaded}} that the shaded jar has no {{org/rocksdb}}
> classes or {{librocksdbjni}} files.
> h3. How to check
> mvn dependency:tree -pl :ozone-filesystem-shaded
> -Dincludes=org.rocksdb:rocksdbjni # empty after the fix
> unzip hadoop-ozone/ozonefs-hadoop3/target/ozone-filesystem-hadoop3*.jar |
> grep -c librocksdbjni # 0
> ls -- hadoop-ozone/ozonefs-hadoop3/target/ozone-filesystem-hadoop3*.jar #
> about 70 MB
>
> {noformat}
> {noformat}
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]