Hi Alex, amazing work! congrats to reach to this point! :)
I need to put my head on all of this, but count on me to be a RM. I think the best thing should be that you be the first RM to try your own development at least for the first time, and then the rest of us will follow you on the next releases. With all this on place we maybe could release once a month or every two months... Thanks for doing this :) Carlos El vie., 8 mar. 2019 a las 1:55, Alex Harui (<aha...@adobe.com.invalid>) escribió: > OK, I've now seen Jenkins perform the steps to build the release > artifacts. Folks interested in Docker-izing the steps are welcome to look > at the jobs on the "Royale Release" tab on the CI server. > http://apacheroyaleci.westus2.cloudapp.azure.com:8080/view/Royale%20Release/ > > These steps assume that the RM can run the basic Maven and Ant build on > the RM's computer. I think that's a fair requirement since all of us on the > PMC need to able to do that to build the RC in order to vote on it. > Jenkins does other tasks like run the Maven release plugin steps. > > Currently that results in binaries on Jenkins that are downloaded to the > RM's computer. These binaries need to be verified by the RM which is the > next phase I will be starting on now. The RM verifies the bits and then > PGP signs them. And then the bits are uploaded off the RM's computer to > Maven Staging or dist.a.o/dev. If that uploading turns out to be a point > of failure, we have the option of having Jenkins upload the big files and > have the RM only upload PGP signature files. Or finding a way for Jenkins > to get the signature files from the RM. The uploads worked fine for me, > but then again, so did the old script's uploads. > > Therefore, once I get the binary verification phase completed, I think > someone other than me should be the RM and try to use these steps to > generate the release and help debug the process for the next RM. So, > please try to carve out some time to be the RM. One advantage of doing > most of the work on Jenkins is that it frees up my computer to do other > things while Jenkins is cranking away. > > I think we're at least a week away from binary verification, maybe two, so > it is time to start thinking about what is going in this release. > > Thanks, > -Alex > > On 3/7/19, 4:15 PM, "Alex Harui" <aha...@adobe.com.INVALID> wrote: > > In case you haven't guessed, I'm testing out Jenkins and its ability > to create the artifacts and send emails. Please ignore any email that > looks like a vote or discuss thread. > > Thanks, > -Alex > > On 2/10/19, 8:44 PM, "Alex Harui" <aha...@adobe.com.INVALID> wrote: > > Om, > > Well, that's what I used for the last release. I'm sure there is > probably some inaccuracy in it. > > That said, I'm abandoning that document and taking a new angle > because that document presumed that the release manager was trying to > create a release on his/her computer. I've given up on that and working on > making releases from a shared computer for the reasons I've stated > upthread. I hope to make some progress on that this week. > > The key difference is that the new angle does not presume that you > have Git SSH and PGP signatures all set up on some computer. I have not > looked into how Docker would handle that. You certainly wouldn't want the > Docker image to contain your SSH or PGP keys/creds. And if the Docker > image doesn't, then that is another stumbling block for future RMs. > > The other key difference is that the old script presumed you could > create the 3 release in 3 huge "easy" steps. We've seen that is only true > for me. So the new angle creates many discrete steps managed by Jenkins. > > So, it is up to you to decide what you want to "Docker-ize". You > can try to Docker-ize the current 3 big steps, so RMs can try to run it on > their systems, but I'd bet they will just faiI due to network issues. I > would be interested in using Docker to make each of these many discrete > steps portable to another server. I'm not going to involve Docker at this > point. My main goal is just to see if I can create a workflow of many > discrete steps that isn't horribly painful. Once we see what these steps > turn out to be, then we can worry about server portability of those steps. > > Either way, we want to know about running Browser+Selenium for > sure. And maybe FlashPlayerDebugger or AIR. I would want to know, for > example, how you debug a failing checkintest in a Docker container. > > Thanks, > -Alex > > > On 2/10/19, 5:18 PM, "OmPrakash Muppirala" <bigosma...@gmail.com> > wrote: > > Alex, > > Just to be clear, I am following the steps from here to try > and setup a > docker container. > > > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Froyale-asjs%2Fwiki%2FRelease-Manager-Notes&data=02%7C01%7Caharui%40adobe.com%7C9a4f659f80694687c3cd08d6a35b378d%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636876009505592396&sdata=rJsHtgUQ%2BH5iQOXlZc%2FCEvtxH0Lm25fGn0wlAe6oqSs%3D&reserved=0 > > Is this doc up to date? > > Thanks, > Om > > On Wed, Feb 6, 2019 at 12:02 PM OmPrakash Muppirala < > bigosma...@gmail.com> > wrote: > > > One approach is to have each step in the process spin up a > docker image. > > And use docker-compose to run each step. All the images can > be made to > > share a common volume where all the artifacts are stored > across steps. > > > > You are right about the networking issue though. Any > network related > > failure that occurs on the host machine will most likely > occur in the > > docker container. Although, I am not clear how we can > guarantee that the > > same issues will not occur on the Jenkins server. I mean, > what is special > > about the Jenkins server that makes it immune to these > networking issues? > > > > Thanks, > > Om > > > > On Wed, Feb 6, 2019 at 11:48 AM Alex Harui > <aha...@adobe.com.invalid> > > wrote: > > > >> Again, I don't know anything about Docker, but before we > spend a lot of > >> time on Docker, I also want to point out that the process > to create a > >> release cannot really be thought of as "one application". > It will be a > >> series of "steps" to run. How many steps depends on > whether we think we > >> can isolate enough stuff via Docker to be able to run > Docker on the RM's > >> computer instead of some shared computer. On a shared > computer there will > >> be dozens of steps because the RM will need to enter > passwords to commit > >> stuff. On a local computer I guess the RM can supply > passwords but I think > >> there will be stopping points where the Maven artifacts are > deployed and > >> the staging repo is closed, and another stopping point for > the vote. It > >> seems like Docker works by downloading dependencies. Given > that the > >> problem the RMs had last time involved downloads and > uploads, why do we > >> think Docker will really solve this for creating releases > on local machines? > >> > >> The reason to do this on a shared machine is so that new > RMs don't have > >> to do as much setup. But then I wonder about the > efficiency of kicking off > >> that many Docker images. Jenkins can manage that already. > Does Docker > >> have some sort of similar Dashboard or would we use Jenkins > to kick off > >> Docker steps? I can't quite picture what is the outermost > >> control/dashboard. > >> > >> -Alex > >> > >> On 2/6/19, 11:03 AM, "Harbs" <harbs.li...@gmail.com> wrote: > >> > >> A quick search turns up this: > >> > >> > >> > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcodingsans.com%2Fblog%2Fselenium-with-docker-testing&data=02%7C01%7Caharui%40adobe.com%7C9a4f659f80694687c3cd08d6a35b378d%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636876009505592396&sdata=fA7bthRUAGXzqJdmK4raheZMEHQ4qyy3NHXjufAF6jc%3D&reserved=0 > >> < > >> > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcodingsans.com%2Fblog%2Fselenium-with-docker-testing&data=02%7C01%7Caharui%40adobe.com%7C9a4f659f80694687c3cd08d6a35b378d%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636876009505592396&sdata=fA7bthRUAGXzqJdmK4raheZMEHQ4qyy3NHXjufAF6jc%3D&reserved=0 > >> > > >> > >> > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.blazemeter.com%2Fblog%2Fhow-to-run-selenium-tests-in-docker&data=02%7C01%7Caharui%40adobe.com%7C9a4f659f80694687c3cd08d6a35b378d%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636876009505592396&sdata=QeJ5h98H2ghmY%2BMoGWKVgYaS6OKiv9BY7Cbu%2FRYh9K0%3D&reserved=0 > >> < > >> > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.blazemeter.com%2Fblog%2Fhow-to-run-selenium-tests-in-docker&data=02%7C01%7Caharui%40adobe.com%7C9a4f659f80694687c3cd08d6a35b378d%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636876009505592396&sdata=QeJ5h98H2ghmY%2BMoGWKVgYaS6OKiv9BY7Cbu%2FRYh9K0%3D&reserved=0 > >> > > >> > >> > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSeleniumHQ%2Fdocker-selenium&data=02%7C01%7Caharui%40adobe.com%7C9a4f659f80694687c3cd08d6a35b378d%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636876009505592396&sdata=pvNt%2B4Spevu9uXaZeX1bV4AKi3p1KVnwND4XZvqlxt0%3D&reserved=0 > >> < > >> > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSeleniumHQ%2Fdocker-selenium&data=02%7C01%7Caharui%40adobe.com%7C9a4f659f80694687c3cd08d6a35b378d%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636876009505592396&sdata=pvNt%2B4Spevu9uXaZeX1bV4AKi3p1KVnwND4XZvqlxt0%3D&reserved=0 > >> > > >> > >> > On Feb 6, 2019, at 8:59 PM, Alex Harui > <aha...@adobe.com.INVALID> > >> wrote: > >> > > >> > Thanks for volunteering to try it Om. IMO, even more > important > >> than Firefox+Flash is SomeBrowser+Selenium. We may also > need to run Adobe > >> AIR's adb. We could probably turn off the Flash tests or > replace Flash > >> with AIR. > >> > > >> > Thanks, > >> > -Alex > >> > > >> > On 2/6/19, 10:54 AM, "Carlos Rovira" < > carlosrov...@apache.org> > >> wrote: > >> > > >> > Hi Om, that would be great! waiting for your > experience with a > >> that! :) > >> > > >> > El mié., 6 feb. 2019 a las 19:31, OmPrakash > Muppirala (< > >> bigosma...@gmail.com>) > >> > escribió: > >> > > >> >> Yes, I agree with Alex about not being able to run > UI out of docker > >> >> containers. I never thought of the checkintests > when I made the > >> >> suggestion. > >> >> > >> >> The firefox-flash image that Yishay pointed out > looks promising. > >> I will > >> >> poke around with it and see if that works for us. > >> >> > >> >> Thanks, > >> >> Om > >> >> > >> >> On Wed, Feb 6, 2019 at 12:41 AM Yishay Weiss < > >> yishayj...@hotmail.com> > >> >> wrote: > >> >> > >> >>> I’ll let Om or someone else with docker experience > tell us if > >> this [1] is > >> >>> relevant. > >> >>> > >> >>> > >> >>> > >> >>> [1] > >> > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhub.docker.com%2Fr%2Fbeli%2Ffirefox-flash%2F&data=02%7C01%7Caharui%40adobe.com%7C9a4f659f80694687c3cd08d6a35b378d%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636876009505592396&sdata=O5WCCm7fcFGjaXpoDum5hMAJXl3JWscFMRMywpBAN1w%3D&reserved=0 > >> >>> > >> >>> > >> >>> > >> >>> ________________________________ > >> >>> From: Alex Harui <aha...@adobe.com.INVALID> > >> >>> Sent: Wednesday, February 6, 2019 10:05:54 AM > >> >>> To: dev@royale.apache.org > >> >>> Subject: Re: 0.9.6 Release > >> >>> > >> >>> I don't anything about docker, but in 15 minutes of > reading I ran > >> into > >> >>> this: > >> >>> > >> >>> > >> >> > >> > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.channelfutures.com%2Fopen-source%2Fwhen-not-to-use-docker-understanding-the-limitations-of-containers&data=02%7C01%7Caharui%40adobe.com%7C9a4f659f80694687c3cd08d6a35b378d%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636876009505602405&sdata=bjIQhi1pOBfBd8jQAc0XmbrsWzlDwSqieUiUoSovnIE%3D&reserved=0 > >> >>> > >> >>> Which says: Docker can't " Run applications with > graphical > >> interfaces". > >> >>> If you want Royale to use Docker for releases, show > that it can > >> run > >> >>> checkintests with Flash and the Browser. Then I > will look into > >> it more. > >> >>> > >> >>> The highest level goal is to make it as easy as > possible for > >> someone to > >> >>> volunteer to be an RM. Any requirement of "install > this (Docker, > >> etc) on > >> >>> your computer" is, IMO, another barrier to entry. > Yeah, RMs will > >> have to > >> >>> have Maven installed and maybe Ant, but you should > already have > >> those > >> >>> installed to be a committer/PMC member. > >> >>> > >> >>> That said, a good takeaway from the Docker idea is > to try to find > >> a way > >> >> to > >> >>> make an "Image" of whatever we end up with on > whatever server we > >> end up > >> >>> using so if the image can be copied and used on > other servers. > >> I'm not > >> >>> exactly sure how to do that with Azure, which hosts > my CI > >> server. I will > >> >>> spend a few more minutes researching that. > >> >>> > >> >>> I could not quickly find any way to get a free VM > on Azure or AWS > >> that > >> >>> isn't a free-trial-start-paying-after-a-year. So, > unless someone > >> comes > >> >> up > >> >>> with a free server we can use "forever", I'm going > to just start > >> with my > >> >>> Azure VM. > >> >>> > >> >>> -Alex > >> >>> > >> >>> On 2/5/19, 10:59 PM, "Carlos Rovira" < > carlosrov...@apache.org> > >> wrote: > >> >>> > >> >>> Hi. > >> >>> > >> >>> the plan sounds very good to me. Just my 2 > thoughts on this: > >> >>> > >> >>> 1.- As I was reading I was thinking as well on > something like > >> Docker > >> >>> and > >> >>> see Om as well thinking on the same. Maybe is > the way to this > >> with > >> >> the > >> >>> actual technology. Seems VMs are stepping out a > bit this days > >> in > >> >> favor > >> >>> of > >> >>> things like Docker. Maybe the same did Git over > Svn, and today > >> Svn is > >> >>> an > >> >>> old remembrance. I must say that I have no > experience with > >> Docker, so > >> >>> doing > >> >>> that will require acquire that knowledge, but > seems it could > >> be worth > >> >>> it. > >> >>> > >> >>> 2.- Maybe is not possible, but I want to propose > to do this > >> work I a > >> >>> separate branch, so it could be in parallel to > other > >> developments. I > >> >>> think > >> >>> work over develop is practical if there's > something tiny that > >> could > >> >> be > >> >>> done > >> >>> in a commit. But as we need more than one, or is > a long > >> process (like > >> >>> this), chances are to make develop branch > unstable and even > >> for some > >> >>> days. > >> >>> I think we should try to avoid that scenario, > and branches are > >> the > >> >> best > >> >>> way. If we do this way, we'll benefit of more > reliable develop > >> >> branch. > >> >>> > >> >>> Thanks and good to know of this plan :) > >> >>> > >> >>> Carlos > >> >>> > >> >>> > >> >>> El mar., 5 feb. 2019 a las 23:19, Harbs (< > >> harbs.li...@gmail.com>) > >> >>> escribió: > >> >>> > >> >>>> I’ve never used Docker myself, but that might be a > good plan. > >> >>>> > >> >>>>> On Feb 6, 2019, at 12:07 AM, OmPrakash Muppirala < > >> >>> bigosma...@gmail.com> > >> >>>> wrote: > >> >>>>> > >> >>>>> I was wondering if we can use docker images to > setup and seal > >> the > >> >>> RM > >> >>>>> environment. Then other RMs simply need to run > the image > >> locally > >> >>> and run > >> >>>>> the release scripts. Might be easier. If folks > like this plan, > >> >> I > >> >>> can > >> >>>> try > >> >>>>> to put something together. > >> >>>>> > >> >>>>> Thanks, > >> >>>>> Om > >> >>>>> > >> >>>>> On Tue, Feb 5, 2019 at 1:40 PM Harbs < > harbs.li...@gmail.com> > >> >>> wrote: > >> >>>>> > >> >>>>>> In a recent discussion, it looks like other > projects have > >> gotten > >> >>>> resources > >> >>>>>> from AWS. > >> >>>>>> > >> >>>>>> Whatever service we use, could setup a “shared” > Royale account > >> >>> that all > >> >>>>>> PMC members could have access to. > >> >>>>>> > >> >>>>>> I don’t know if there’s some way we could > leverage Gitlab’s > >> >>> integration > >> >>>>>> pipelines > >> >>> > >> >> > >> > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.gitlab.com%2Fee%2Fci%2FREADME.html&data=02%7C01%7Caharui%40adobe.com%7C9a4f659f80694687c3cd08d6a35b378d%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636876009505602405&sdata=ZrqmL9%2B0oXasDm4aDW2q8K%2F%2BEkSc17zQerzJ4m%2BM7lU%3D&reserved=0 > >> >>> < > >> >>>>>> > >> >>> > >> >> > >> > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.gitlab.com%2Fee%2Fci%2FREADME.html&data=02%7C01%7Caharui%40adobe.com%7C9a4f659f80694687c3cd08d6a35b378d%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636876009505602405&sdata=ZrqmL9%2B0oXasDm4aDW2q8K%2F%2BEkSc17zQerzJ4m%2BM7lU%3D&reserved=0 > >> >>>> > >> >>>>>> > >> >>>>>>> On Feb 5, 2019, at 11:33 PM, Alex Harui > >> >> <aha...@adobe.com.INVALID > >> >>>> > >> >>>>>> wrote: > >> >>>>>>> > >> >>>>>>> Well, the big hole in this plan is that I think > we have to use > >> >>>> someone's > >> >>>>>> personal VM account (in this case, mine). I > can't think of a > >> >> way > >> >>> we can > >> >>>>>> run interactive commands like git push on builds@. > But that > >> >>> reminds me > >> >>>>>> to go see what are current options are for > free/cheap compute > >> >>> servers. > >> >>>>>>> > >> >>>>>>> On 2/5/19, 1:20 PM, "Piotr Zarzycki" < > >> >> piotrzarzyck...@gmail.com> > >> >>>> wrote: > >> >>>>>>> > >> >>>>>>> Sounds like best plan ever. Using the same PC > by everyone is > >> >>> awesome! > >> >>>>>>> > >> >>>>>>> On Tue, Feb 5, 2019, 8:39 PM Harbs < > harbs.li...@gmail.com > >> >>> wrote: > >> >>>>>>> > >> >>>>>>>> A big +1 from me! > >> >>>>>>>> > >> >>>>>>>> Looking forward! > >> >>>>>>>> > >> >>>>>>>>> On Feb 5, 2019, at 9:34 PM, Alex Harui > >> >>> <aha...@adobe.com.INVALID> > >> >>>>>> wrote: > >> >>>>>>>>> > >> >>>>>>>>> Hi, > >> >>>>>>>>> > >> >>>>>>>>> We are coming up on 3 months since 0.9.4. I > have finished > >> >> the > >> >>>> changes > >> >>>>>>>> to get production Royale modules to work in > Tour De Flex. > >> >> Lots > >> >>> of > >> >>>> other > >> >>>>>>>> good changes have been contributed. > >> >>>>>>>>> > >> >>>>>>>>> There were emails around the 0.9.4 release > about others > >> >>> stepping up > >> >>>> to > >> >>>>>>>> cut the next release, but that hasn't > happened. I tried and > >> >>> failed to > >> >>>>>> get > >> >>>>>>>> Apache Infra to allow us to run our release > packaging on the > >> >>> Jenkins > >> >>>>>>>> servers. They felt there were too many > security concerns > >> with > >> >>> having > >> >>>>>> the > >> >>>>>>>> servers push changes to Git and PGP sign > artifacts. > >> >>>>>>>>> > >> >>>>>>>>> However, we MUST find a way for other RMs to > be successful. > >> >>> There is > >> >>>>>> no > >> >>>>>>>> way I should or want to be the only RM. But I > have an idea > >> >> that > >> >>>>>> involves > >> >>>>>>>> creating a long list of Jenkins jobs on my CI > server that add > >> >>> up to > >> >>>> the > >> >>>>>>>> release. The RM would log into Jenkins and > run some job > >> >> titled > >> >>>> "Apache > >> >>>>>>>> Royale Release Step 1", then wait for an email > indicating > >> that > >> >>> it > >> >>>>>> completed > >> >>>>>>>> and follow instructions in the email, such as > logging into > >> the > >> >>> CI > >> >>>> server > >> >>>>>>>> via Remote Desktop, opening a command prompt > and running "git > >> >>> push" > >> >>>> and > >> >>>>>>>> entering his/her username and password. Then > run the next > >> job > >> >>> and so > >> >>>>>> on. > >> >>>>>>>> There will be a point where the RM has to > download the build > >> >>>> artifacts, > >> >>>>>>>> verify them, then PGP sign them, and upload > them. That will > >> >> be > >> >>> a > >> >>>> likely > >> >>>>>>>> point of failure, but that step should be a > single Maven > >> >>> command and > >> >>>>>> thus > >> >>>>>>>> can be restarted until it finally succeeds. > Then more > >> Jenkins > >> >>> jobs > >> >>>>>> will be > >> >>>>>>>> run. But if this works then folks won't have > to setup their > >> >>> computers > >> >>>>>> to > >> >>>>>>>> be an RM. > >> >>>>>>>>> > >> >>>>>>>>> So, prepare for lots of commits and reverts > as I try to put > >> >>> this > >> >>>>>>>> together. I'm guessing it won't actually work > until the last > >> >>> week of > >> >>>>>>>> February at the earliest so there is no big > rush to get other > >> >>> stuff in > >> >>>>>> for > >> >>>>>>>> this release, and if we get it to work, > hopefully we'll > >> >> release > >> >>> more > >> >>>>>> often > >> >>>>>>>> with other folks being the RM. > >> >>>>>>>>> > >> >>>>>>>>> -Alex > >> >>>>>>>>> > >> >>>>>>>> > >> >>>>>>>> > >> >>>>>>> > >> >>>>>>> > >> >>>>>> > >> >>>>>> > >> >>>> > >> >>>> > >> >>> > >> >>> -- > >> >>> Carlos Rovira > >> >>> > >> >>> > >> >> > >> > https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fabout.me%2Fcarlosrovira&data=02%7C01%7Caharui%40adobe.com%7C9a4f659f80694687c3cd08d6a35b378d%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636876009505602405&sdata=egeH1lqxrifO9sWYd0PO67pKuQP6Riv0N2tEDf9rriA%3D&reserved=0 > >> >>> > >> >>> > >> >>> > >> >> > >> > > >> > > >> > -- > >> > Carlos Rovira > >> > > >> > https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fabout.me%2Fcarlosrovira&data=02%7C01%7Caharui%40adobe.com%7C9a4f659f80694687c3cd08d6a35b378d%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636876009505602405&sdata=egeH1lqxrifO9sWYd0PO67pKuQP6Riv0N2tEDf9rriA%3D&reserved=0 > >> > > >> > > >> > >> > >> > >> > > > > > > > -- Carlos Rovira http://about.me/carlosrovira
