The requirements for the scripts are:
Filemanager as a component must work without docker, and without
RADIX, so any changes to that core script need to satisfy that.
Additionally Filemanager has no dependencies on workflow, other
envs, or resource, etc., so any updates to the FM script in the core
FM component would need to respect that
RADIX on the other hand, depends on everything, env variables, everything
else, the core package. Given that, it makes sense to have its filemgr script
be different and part of something bigger.
Does this make sense?
Cheers,
Chris
From: Imesha Sudasingha <ime...@apache.org>
Reply-To: <dev@oodt.apache.org>
Date: Wednesday, June 5, 2019 at 11:08 PM
To: dev <dev@oodt.apache.org>, Chris Mattmann <mattm...@apache.org>
Subject: Re: [GSOC] OODT docker build meeting summary
Hi Yasith,
Thanks for the meeting notes. Regarding those two questions,
1. Yes, I have noticed that those scripts are out of sync. But, we cannot
make those two even directly since, the script within Radix is supposed to
be run by another shell script which exports some variables like OODT_HOME,
RUN_JAVA. In contrast, shell script within filmgr module is supposed to be
run in standalone mode. Therefore, if we are to sync those two scripts,
only possibility is to add any missing lines and etc. @Chris Mattmann
<mattm...@apache.org> what is your opinion on this?
2. They should definitely be updated. Could you create Jira issues for
those pom files (one issue containing all pom files will be enough)? It
will be great if you can send a fix for that as well.
Thanks,
Imesha
On Tue, 4 Jun 2019 at 22:20, Jayawardana, Yasith <yas...@cs.odu.edu> wrote:
Hi All,
Here are the meeting minutes from the discussion today with Imesha, and
the discussion last week with Tom.
05/28 - GSOC Meeting [with Tom] (9 AM EST)
* Status
* I have created the DMB repository before the meeting as first
step, and plan to add the dockerfile-maven plugin next.
* Discussion
* Set default docker image name to project build name and the
default tag to project version
* Run the shell script for file manager in docker container and see
fix errors.
* Also create a diagram indicating which components depend on the
others so that others can benefit.
* Goal for the week is to get to a stage where dockerfile maven
generates the docker image, and filemgr can run in it without errors
06/04 - GSOC Meeting [with Imesha] (9 AM EST)
* Status
* I created a dockerfile for filemgr and made a few changes to the
filemgr script, so that filemgr start works inside docker as well. Now the
maven build process generates a docker image from the dockerfile at the
package phase, and I’m able to start it without encountering any issues.
* Discussion
* Instead of running the shell script, run the command directly in
Dockerfile. Makes things easier.
* For the development purposes, create the docker images on the
default build profile itself. Later move them to a separate profile.
* Once the image works and confirmed running well, discuss with
others on which directories should be mounted in volumes.
There are also a few concerns that I forgot to discuss on today’s meeting
which I want to mention.
* I saw that filemgr shell script inside radix is different from the
one in core distribution. Is this intentional or is it a bug?
* I saw some plugin versions in pom files are beta (especially
maven-assembly-plugin). Shouldn’t these be updated to non-beta versions?
Appreciate any suggestions on these.
Best Regards,
Yasith Jayawardana
On May 27, 2019, at 10:04 PM, Jayawardana, Yasith <yas...@cs.odu.edu
<mailto:yas...@cs.odu.edu>> wrote:
[MEETING NOTES - 05/21/2019]
* As the first step of the project, create a new repository from the
RADIX build.
* Customize it such that it creates Docker Images at “mvn install”, and
publishes them to DockerHub at “mvn deploy”.
* To achieve this, the first component that should be containerized is
OODT file manager, as it contains the least dependencies on other
components of OODT.
* Maven assembly plugin is used at the "mvn build" stage. Therefore,
the docker image creation should happen in a later stage. i.e. “mvn
install” stage.
* Use the tar.gz file created from the maven assembly plugin as the
source for generating the docker image for that component.
* The docker image creation should not be enabled by default, and
should be designed as an optional output. User should be able to configure
the credentials of DockerHub, etc somewhere in the build step.
Apologies for taking long to post this update.
[PROGRESS UPDATE]
So far, I created a new repository (
https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fyasithmilinda%2Foodt-dmb&data=02%7C01%7Cyasith%40cs.odu.edu%7C562ed13a48344223267608d6e310e8ac%7C48bf86e811a24b8a8cb368d8be2227f3%7C0%7C0%7C636946059106151479&sdata=8DNciBssR3lnur1CEMiXuTJjqQxl1NZUd7XL6iN3cPg%3D&reserved=0)
to implement the project and add the additional build steps needed to
create/publish docker images. As the implementation becomes stable, the
changed/added files will be added/modified in OODT source code, and this
repo could be used as an example implementation of it.
If there’s anything that I've misunderstood and/or needs to be changed, do
let me know. Thanks!
Best,
Yasith
On May 14, 2019, at 11:17 AM, Tom Barber <t...@spicule.co.uk<mailto:
t...@spicule.co.uk><mailto:t...@spicule.co.uk>> wrote:
Yasith, to get you off to a good start in terms of open source development.
Can you please file the issues you expect to work on in the short term in
Jira so we can track them. If you do it it’ll ensure that you have enough
knowledge rather than us doing it and them not making sense.
Tom
On 14 May 2019 at 15:12:43, Tom Barber (t...@spicule.co.uk<mailto:
t...@spicule.co.uk><mailto:t...@spicule.co.uk>) wrote:
Just a quick recap from our call with Yasith earlier today:
Had a good chat with Yasith about how RADIX works and how people build out
OODT applications from RADIX (
https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FOODT%2FRADiX%2BPowered%2BBy%2BOODT&data=02%7C01%7Cyasith%40cs.odu.edu%7C562ed13a48344223267608d6e310e8ac%7C48bf86e811a24b8a8cb368d8be2227f3%7C0%7C0%7C636946059106151479&sdata=9hbMVRs1dsT2PB08QFS69IYc88cgKWjzSTITUn%2B9Njg%3D&reserved=0
)
General Docker build plan is as follows:
* Users can deploy a standard RADIX build and optionally choose the Docker
profile at build time to build docker images from it
* Docker will be optional not standard
* RADIX will allow users to define the image tag names for easy tagging and
pushing to docker hub or similar
* In the GSOC version we will focus on single deployments of each component
not clustered up using ZK or similar we will stick that in the TODO list
Once built the end target is Kubernetes, both MiniKube and full Kubernetes.
Docker Compose may be used for interim testing and validation.
A few deploy methods exist for validation:
Standard deploy yams for K8S
Helm Chart
Or via a custom Kubernetes Operator similar to ArangoDB (
https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Farangodb%2Fkube-arangodb&data=02%7C01%7Cyasith%40cs.odu.edu%7C562ed13a48344223267608d6e310e8ac%7C48bf86e811a24b8a8cb368d8be2227f3%7C0%7C0%7C636946059106151479&sdata=mt2pG%2BmMTcAd2FJuXtn%2FHwro090o4TBb6Um%2Fi1rzAvA%3D&reserved=0
)
--
Spicule Limited is registered in England & Wales. Company Number:
09954122. Registered office: First Floor, Telecom House, 125-135 Preston
Road, Brighton, England, BN1 6AF. VAT No. 251478891.
All engagements
are subject to Spicule Terms and Conditions of Business. This email and its
contents are intended solely for the individual to whom it is addressed and
may contain information that is confidential, privileged or otherwise
protected from disclosure, distributing or copying. Any views or opinions
presented in this email are solely those of the author and do not
necessarily represent those of Spicule Limited. The company accepts no
liability for any damage caused by any virus transmitted by this email. If
you have received this message in error, please notify us immediately by
reply email before deleting it from your system. Service of legal notice
cannot be effected on Spicule Limited by email.
