[
https://issues.apache.org/jira/browse/HDDS-1458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16831724#comment-16831724
]
Eric Yang commented on HDDS-1458:
---------------------------------
[~elek] I will rebase the patch to current trunk today.
{quote}Can you please describe what is the problem exactly? I have some
concerns to create a docker image with each build. It's time and space
consuming. {quote}
The current ozone docker image is not easily transportable. A well design
docker image should have no host level binary dependency. In order to share
the ozone docker image to another host, the ozone tarball location on the host
must be copied to a second host in order to reproducible the in-sync state as
the docker image from the first host, and use docker-compose to lock host level
binaries and docker image together to produce a functional system. This is not
intended approach to use Docker image.
If there is no change to files that is used to build docker image layer, it
will simply use a reference count instead of regenerating entire layer. Each
line in the Dockerfile produce a docker image immutable layer, and docker is
pretty good to cache the output without having to rebuild everything from
scratch. A well designed docker image build process may take minutes in it's
first time build, but subsequence build only takes sub-seconds to perform.
Unless the layers have changed, otherwise they do not take up more space than a
reference count. Ozone tar stitching is same as building a layer of Docker but
at host level. Since any kind of system test, we are already doing tar/untar
operations.
The cost of building a Ozone image is same as running tar expand, the cost can
easily be justified. The misconception of docker build process is expensive,
it really depends on how the code is structured. If the high frequency changes
are placed toward end of the image creation, then the time spend in docker
build can be really small. We can always skip docker image build process with
-DskipDocker. This is similiar to -DskipShade for people that don't work in
those areas.
{quote}I believe it's more effective to use just the hadoop-runner image and
mount the built artifact. I would like to understand the problem in more
details to find an effective solution. Wouldn't be enough to use the compose
files form the target folder?{quote}
Docker image are designed to host binary executables as layers of immutable
file system changes. This provides predictable out come when binary are
swapped out between container instances. When binary executables are outside
of docker container, the stability of the docker instance depends on the
external mounted binaries. By mounting external executable binaries, it become
less reproducible because it has heavy dependency on external mount point state
being in sync with container image states. Standalone docker image are more
effective way to share containers than docker-compose stitch host level
binaries together with a empty container.
> Create a maven profile to run fault injection tests
> ---------------------------------------------------
>
> Key: HDDS-1458
> URL: https://issues.apache.org/jira/browse/HDDS-1458
> Project: Hadoop Distributed Data Store
> Issue Type: Test
> Reporter: Eric Yang
> Assignee: Eric Yang
> Priority: Major
> Attachments: HDDS-1458.001.patch, HDDS-1458.002.patch
>
>
> Some fault injection tests have been written using blockade. It would be
> nice to have ability to start docker compose and exercise the blockade test
> cases against Ozone docker containers, and generate reports. This is
> optional integration tests to catch race conditions and fault tolerance
> defects.
> We can introduce a profile with id: it (short for integration tests). This
> will launch docker compose via maven-exec-plugin and run blockade to simulate
> container failures and timeout.
> Usage command:
> {code}
> mvn clean verify -Pit
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
