Thank you for a proposal - to have a containerised CI solution will allow us to test with more version of interpreters and real clusters, as well as reduce build times, which is awesome.
I wish we could also keep the simplicity of setup using `.travis.yaml` rather then job configuration details in CI system, but that is minor. Are there any plans to donate this work to ASF? If so, we could keep it in i.e ./dev/ci/ after you think it's ready So please keep us posted on the progress - indeed it would be great to see it working on some repo. As for infrastructure, it might be quite challenging to setup builds.apache.org that way (I can make a research after the New Year), so we might need to think more, what are the options. On Mon, Dec 21, 2015 at 4:54 PM, Hyung Sung Shim <[email protected]> wrote: > Thank you for your feedback Dear Jongyoul Lee. > > 2015-12-20 23:47 GMT-08:00 Jongyoul Lee <[email protected]>: > > > Looks good!! This will reduce Zeppelin's testing issue. > > > > On Mon, Dec 21, 2015 at 4:42 PM, Hyung Sung Shim <[email protected]> > > wrote: > > > > > Thank you for your reply. moon. :) > > > > > > Now we are testing the CI in private. > > > So CI test environments for anybody is going to be prepared. > > > I'll let you know when CI test environments is ready. > > > > > > Thanks. > > > > > > 2015-12-20 22:16 GMT-08:00 moon soo Lee <[email protected]>: > > > > > > > Thanks for proposing a new CI system. > > > > It will definitely improves Zeppelin development. > > > > > > > > Is there any github repository that configured with this proposed CI > > > > system? so anyone can make a pullrequest there and test it. That > would > > be > > > > really helpful. > > > > > > > > There're jenkins that managed by ASF. https://ci.apache.org/#jenkins > . > > > > But not sure this CI can be run on ASF jenkins, while it requires > > docker. > > > > If someone can clarify, that would be appreciated. > > > > > > > > Thanks, > > > > moon > > > > > > > > On Mon, Dec 21, 2015 at 11:59 AM Amos B. Elberg < > [email protected] > > > > > > > wrote: > > > > > > > > > I think I know a PR you could use to test it out. :) > > > > > > > > > > From: Hyung Sung Shim <[email protected]> > > > > > Reply: [email protected] < > > > > > [email protected]> > > > > > Date: December 20, 2015 at 9:56:31 PM > > > > > To: [email protected] < > > > [email protected] > > > > > > > > > > Subject: [DISCUSS] CI system for zeppelin > > > > > > > > > > Dear devs. > > > > > > > > > > I'd like to propose new CI system for the Zeppelin project which is > > > > testing > > > > > in NFLabs. > > > > > > > > > > I knew in some cases current build system (travis) can not help > > > > developping > > > > > Zeppelin. > > > > > 1. It can't clsasloading test at the spark cluster because of now > CI > > > > > environment is not the actual environment. > > > > > (zeppelin and Spark is running in the same host.) > > > > > 2. It can't test on the various Operating System and JDK version > > > > > environments. (only build in the travis host OS and single version > of > > > > JDK) > > > > > 3. It can't test on the various spark cluster environments. (now > only > > > > build > > > > > with the spark standalone.) > > > > > 4. It can't parallel test because of build time. (now spark > > standalone > > > > test > > > > > takes 40 minutes.) > > > > > > > > > > *. The proposed CI system can > > > > > 1. clsasloading test at the spark cluster in actual environment by > > > > testing > > > > > Zeppelin in the docker container completely separated with Spark > > > cluster. > > > > > (Spark cluster also runs on docker.) > > > > > 2. test on the various Operating System and JDK version > environments. > > > > > 3. test on the various spark cluster environments. > > > > > - now we build zeppelin with the spark standalone and yarn and > mesos > > > > > backends. > > > > > 4. run parallel build > > > > > - It takes less than 20 minutes when we build the zeppelin with the > > > spark > > > > > standalone and yarn and mesos backends concurrently. > > > > > > > > > > *. Build system introduction. > > > > > Build system is simply composed of the Jenkins and Docker and some > > > > scripts. > > > > > You can get the informations (how to run and how to add new build > > > > features > > > > > etc..) here(https://github.com/NFLabs/zeppelin-ci). > > > > > > > > > > *. Further discussion > > > > > 1. How this proposed CI system can integrated to current Zeppelin > > > github > > > > > repository (hook). > > > > > 2. To run this CI system, we'll need infrastructrure. > > > > > > > > > > I hope it helps both zeppelin developers and users. > > > > > > > > > > Please share your opinion. > > > > > Thanks in advance! > > > > > > > > > > > > > > > > > > > > -- > > 이종열, Jongyoul Lee, 李宗烈 > > http://madeng.net > > >
