Makes sense to me. I will set parallel executors back to 1. And start working on a new Jenkins instance.
Thanks, Om On Sat, Apr 18, 2020, 12:28 AM Alex Harui <[email protected]> wrote: > Are you saying that Java, Ant and Maven do not know how to distribute work > to different cores? > > To repeat my main concern: our Jenkins jobs often use shared resources. > If you want to take the time to fix that, feel free, but I do not know how > you are going to keep two jobs from both trying to launch Flash Player > Debugger at the same time and open the same socket to communicate with it, > or try to parse the flashlog.txt that they have both written to. > > The royale-asjs job will use the royale-compiler workspace to get the > compiler. If the compiler is building, the jars are not there and the > royale-asjs build fails. > > There is also only so much memory and some of our jobs use all of it and > I'm not sure if jobs slowed down due to paging memory and writing/reading > from disk, even if it is an SSD. > > I'm done for tonight, but that's what I was seeing last time I tried to > have both master and an agent running jobs in parallel. > > Thanks, > -Alex > > On 4/17/20, 11:52 PM, "OmPrakash Muppirala" <[email protected]> wrote: > > A typical setup would be to have 0 executors on master and at least n > number of executors on each agent, where n = number of cpu cores. > I see that the current VM has 2 cores. So, I started with setting the > num > executors on the current agent to 2. > > This should improve things quite a bit. If this doesn't work, I will > create a new VM and make it the slave agent of the current Jenkins > master. > > > Thanks, > Om > > On Fri, Apr 17, 2020 at 11:31 PM Alex Harui <[email protected]> > wrote: > > > On your VM or mine? I don't think mine can handle it. I used to > run jobs > > of master and agent and we got lots of failures and jobs took > forever. If > > two jobs are trying to run tests with the shared Flash Player > Debugger, for > > example, you won't get consistent results. > > > > -Alex > > > > On 4/17/20, 11:23 PM, "OmPrakash Muppirala" <[email protected]> > wrote: > > > > I just realized that we have only one Agent (node) listed under > > Jenkins. I > > am going to add at least one more node to start with. > > > > Thanks, > > Om > > > > On Thu, Apr 16, 2020 at 9:58 AM Alex Harui > <[email protected]> > > wrote: > > > > > I think I followed the instructions and enabled SSH. > > > > > > On 4/16/20, 9:37 AM, "Yishay Weiss" <[email protected]> > wrote: > > > > > > I haven’t gotten it to work. Could be ssh needs to be > enabled on > > the > > > VM [1]. This is nice to have but not essential. So Alex, let > me know > > if you > > > want to try that, otherwise I’ll suspend my efforts on this. > > > > > > [1] > > > > > > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmedium.com%2Ftechinpieces%2Fpractical-azure-how-to-enable-ssh-on-azure-vm-84d8fba8103e&data=02%7C01%7Caharui%40adobe.com%7C833f9530b95943d3288908d7e3651005%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637227895512817076&sdata=hxkdbHxBDgkPwstUWJME54TKz9R2X4RdeWpaVzqHQ4g%3D&reserved=0 > > > > > > From: Yishay Weiss<mailto:[email protected]> > > > Sent: Wednesday, April 15, 2020 5:09 PM > > > To: [email protected]<mailto:[email protected]> > > > Subject: RE: Prioritize Release Jobs on CI > > > > > > I’d like to try and install an ssh server. Can anyone > think of a > > > reason not to do that? I’m asking in terms of security, and if > there > > might > > > be a problem because it’s a VM. > > > > > > Thanks. > > > > > > From: Yishay Weiss<mailto:[email protected]> > > > Sent: Monday, April 13, 2020 10:02 AM > > > To: [email protected]<mailto:[email protected]> > > > Subject: RE: Prioritize Release Jobs on CI > > > > > > Related, as most (all?) of the instructions don’t require > > anything but > > > a shell wouldn’t it be easier and faster to access the machine > with > > ssh > > > [1], rather than using remote desktop? > > > > > > [1] > > > > > > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fwindows-server%2Fadministration%2Fopenssh%2Fopenssh_install_firstuse&data=02%7C01%7Caharui%40adobe.com%7C833f9530b95943d3288908d7e3651005%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637227895512817076&sdata=7xKvZszvpz7FovUSvShKXfYqh1b7ulWgW0ZKUkwP%2B%2F4%3D&reserved=0 > > > > > > From: Yishay Weiss<mailto:[email protected]> > > > Sent: Monday, April 13, 2020 9:55 AM > > > To: [email protected]<mailto:[email protected]> > > > Subject: RE: Prioritize Release Jobs on CI > > > > > > Can’t you do that with remote desktop? > > > > > > ________________________________ > > > From: OmPrakash Muppirala <[email protected]> > > > Sent: Monday, April 13, 2020 9:51:25 AM > > > To: Apache Royale Development <[email protected]> > > > Subject: Re: Prioritize Release Jobs on CI > > > > > > Alex, > > > > > > I believe this is your personal Azure subscription right? > Any > > chance > > > I can > > > get access to it so I can try to copy the image out the > machine > > and > > > redeploy it somewhere else? > > > > > > Thanks, > > > Om > > > > > > On Sun, Apr 12, 2020 at 6:18 PM OmPrakash Muppirala < > > > [email protected]> > > > wrote: > > > > > > > I can help set this up on Azure. Give me some time to > work > > out the > > > > details? > > > > > > > > Thanks, > > > > Om > > > > > > > > On Sun, Apr 12, 2020, 11:49 AM Harbs < > [email protected]> > > wrote: > > > > > > > >> OK. Good to know. > > > >> > > > >> > On Apr 12, 2020, at 9:45 PM, Alex Harui > > <[email protected] > > > > > > > >> wrote: > > > >> > > > > >> > FWIW, I just looked and the longest job in the release > > steps so > > > far is > > > >> 8 minutes. Most are under 2 minutes. There might be > jobs > > later > > > that take > > > >> longer that we haven't run yet. IMO, the issue isn't > speed > > of the > > > machine, > > > >> it is just that we are sharing the machine with longer > jobs > > (1 hour > > > for > > > >> TourDeFlexMigration). And again, the machine will be > idle for > > > stretches of > > > >> time while the RM verifies artifacts after each step. > > > >> > > > > >> > -Alex > > > >> > > > > >> > On 4/12/20, 11:32 AM, "Harbs" <[email protected] > > <mailto: > > > >> [email protected]>> wrote: > > > >> > > > > >> > Fair enough. > > > >> > > > > >> > I expect builds to be somewhere between 10 and 20 > times > > faster > > > on a > > > >> powerful machine. > > > >> > > > > >> > Yeah. It’s probably going to be a bit of work > changing > > the > > > server, > > > >> but probably worth it in the long run. > > > >> > > > > >> > I think I’ll try this when I do the next release > unless > > Yishay > > > wants > > > >> to work with me on this for this release — but I’m not > going > > to be > > > able to > > > >> help until after Passover (i.e. next week). > > > >> > > > > >> > Thanks, > > > >> > Harbs > > > >> > > > > >> >> On Apr 12, 2020, at 9:22 PM, Alex Harui > > <[email protected] > > > > > > > >> wrote: > > > >> >> > > > >> >> The Azure portal says: Standard F2s_v2 (2 vcpus, 4 > GiB > > memory) > > > >> >> > > > >> >> I think I am reading changes to the build process in > your > > > >> suggestions. I do not really want to spend more of my > time > > on this > > > >> process. But if you want to do the work, that's fine > with me. > > > >> >> > > > >> >> -Alex > > > >> >> > > > >> >> On 4/12/20, 10:57 AM, "Harbs" <[email protected] > > <mailto: > > > >> [email protected]> <mailto:[email protected] > <mailto: > > > >> [email protected]>>> wrote: > > > >> >> > > > >> >> What kind of horsepower is included in the free > Azure > > account? > > > >> >> > > > >> >> The server I mentioned builds (considerably) > faster than > > my own > > > >> local machine. The ci server seems to build many times > slower. > > > >> >> > > > >> >> One thing we can do to minimize running server time > > would be to > > > >> transfer the artifacts to storage instead of keeping > them on > > the > > > server. On > > > >> AWS, I’d probably use S3. Not sure what the similar > service on > > > Azure is > > > >> called. > > > >> >> > > > >> >>> On Apr 12, 2020, at 8:26 PM, Alex Harui > > > <[email protected] > > > >> <mailto:[email protected]>> wrote: > > > >> >>> > > > >> >>> OK, that's pretty much how I understand Azure as > well. > > The key > > > thing > > > >> is that "running" includes time where the CI server is > not > > running > > > any > > > >> Jenkins jobs. The CI Server steps might take only a few > > hours of > > > actual > > > >> server time, but there is time where the RM is verifying > > artifacts > > > locally > > > >> so you'd be paying for that or the RM would have to keep > > shutting > > > down and > > > >> restarting. > > > >> >>> > > > >> >>> Seems like it would be cheaper/simpler to get the > free > > MSDN > > > account > > > >> and leave it running. > > > >> >>> > > > >> >>> -Alex > > > >> >>> > > > >> >>> On 4/12/20, 10:15 AM, "Harbs" < > [email protected] > > <mailto: > > > >> [email protected]>> wrote: > > > >> >>> > > > >> >>> My experience is with AWS. > > > >> >>> > > > >> >>> I assume Microsoft has similar offerings, but I > don’t > > have > > > >> experience with Azure. > > > >> >>> > > > >> >>> AWS has on-demand EC2 instances which you pay for > only > > the > > > actual > > > >> time that they are running.[1] > > > >> >>> > > > >> >>> Instances can be started and stopped via command > line > > (or via > > > the > > > >> web interface) as long as you have valid credentials to > do so. > > > >> >>> > > > >> >>> For example: an m5.4xlarge instance has 16 cores > and > > costs > > > about > > > >> $1.5 per hour. On a machine like that, a full build > would > > probably > > > take > > > >> less than 10 minutes. It’s probably possible to do a > full > > release > > > with only > > > >> a few hours of server time. > > > >> >>> > > > >> >>> Leaving a server like that running all the time > would get > > > expensive, > > > >> but if it’s just spun up for releases, you’d get very > fast > > builds > > > at a > > > >> reasonable price. > > > >> >>> > > > >> >>> I’d be happy to pay $10-$50 (and possibly more) per > > release to > > > make > > > >> the release process painless for the RM. > > > >> >>> > > > >> >>> [1] > > > >> > > > > > > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Faws.amazon.com%2Fec2%2Fpricing%2Fon-demand%2F&data=02%7C01%7Caharui%40adobe.com%7C833f9530b95943d3288908d7e3651005%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637227895512817076&sdata=BZzHQVAkum497LTo%2BiTThqSgnfPERqSgklluj8QrLKE%3D&reserved=0 > > > >> < > > > >> > > > > > > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Faws.amazon.com%2Fec2%2Fpricing%2Fon-demand%2F&data=02%7C01%7Caharui%40adobe.com%7C833f9530b95943d3288908d7e3651005%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637227895512822068&sdata=CdCoGjKmHETJNgapXtJy4dnAjumpJE7fSEU0FhiN6g8%3D&reserved=0 > > > >> >< > > > >> > > > > > > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Faws.amazon.com%2Fec2%2Fpricing%2Fon-demand%2F&data=02%7C01%7Caharui%40adobe.com%7C833f9530b95943d3288908d7e3651005%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637227895512822068&sdata=CdCoGjKmHETJNgapXtJy4dnAjumpJE7fSEU0FhiN6g8%3D&reserved=0 > > > >> < > > > >> > > > > > > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Faws.amazon.com%2Fec2%2Fpricing%2Fon-demand%2F&data=02%7C01%7Caharui%40adobe.com%7C833f9530b95943d3288908d7e3651005%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637227895512822068&sdata=CdCoGjKmHETJNgapXtJy4dnAjumpJE7fSEU0FhiN6g8%3D&reserved=0 > > > >> >>< > > > >> > > > > > > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Faws.amazon.com%2Fec2%2Fpricing%2Fon-demand%2F&data=02%7C01%7Caharui%40adobe.com%7C833f9530b95943d3288908d7e3651005%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637227895512822068&sdata=CdCoGjKmHETJNgapXtJy4dnAjumpJE7fSEU0FhiN6g8%3D&reserved=0 > > > >> < > > > >> > > > > > > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Faws.amazon.com%2Fec2%2Fpricing%2Fon-demand%2F&data=02%7C01%7Caharui%40adobe.com%7C833f9530b95943d3288908d7e3651005%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637227895512827061&sdata=71%2BpJHsGmOMC%2Bv9lQJdu00Rx8X20SXW5XPKRByL6ftY%3D&reserved=0 > > > >> >< > > > >> > > > > > > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Faws.amazon.com%2Fec2%2Fpricing%2Fon-demand%2F&data=02%7C01%7Caharui%40adobe.com%7C833f9530b95943d3288908d7e3651005%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637227895512827061&sdata=71%2BpJHsGmOMC%2Bv9lQJdu00Rx8X20SXW5XPKRByL6ftY%3D&reserved=0 > > > >> < > > > >> > > > > > > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Faws.amazon.com%2Fec2%2Fpricing%2Fon-demand%2F&data=02%7C01%7Caharui%40adobe.com%7C833f9530b95943d3288908d7e3651005%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637227895512827061&sdata=71%2BpJHsGmOMC%2Bv9lQJdu00Rx8X20SXW5XPKRByL6ftY%3D&reserved=0 > > > >> >>> > > > >> >>> > > > >> >>>> On Apr 12, 2020, at 7:45 PM, Alex Harui > > > <[email protected]> > > > >> wrote: > > > >> >>>> > > > >> >>>> I'm not very experienced with spinning up > servers. The > > CI > > > server we > > > >> are using is effectively free, based on a generous > donation > > from > > > Microsoft > > > >> of MSDN accounts to ASF committers. So I leave it up > 24/7, > > and > > > share the > > > >> RDP access on private@. I think any other ASF > committer > > could do > > > the > > > >> same. IIRC, if that server actually is stopped, I have > to > > use my > > > personal > > > >> (unshared) MSDN credentials to start it again. AIUI, > if I > > > actually paid > > > >> for the server, it would cost me to leave it running > even if > > it > > > didn't run > > > >> jobs between releases. > > > >> >>>> > > > >> >>>> Is that what you are basically saying? I think it > might > > be > > > best if > > > >> another committer got a CI server going via the MS > donation > > and > > > could leave > > > >> it up 24/7. > > > >> >>>> > > > >> >>>> -Alex > > > >> >>>> > > > >> >>>> On 4/12/20, 9:28 AM, "Harbs" < > [email protected]> > > wrote: > > > >> >>>> > > > >> >>>> I’m willing to do this. > > > >> >>>> > > > >> >>>> Considering that the release will be run > infrequently, it > > > should be > > > >> doable to have a relatively powerful server that could > be > > spun up on > > > >> demand. This is something I have setup for my own > releases. > > > >> >>>> > > > >> >>>> The only complication would be that each RM would > need > > valid > > > >> credentials to spin up the server. > > > >> >>>> > > > >> >>>> Harbs > > > >> >>>> > > > >> >>>>> On Apr 12, 2020, at 7:10 PM, Alex Harui > > > <[email protected]> > > > >> wrote: > > > >> >>>>> > > > >> >>>>> A better solution, IMO, is for someone else to > offer up > > a CI > > > server > > > >> only for release jobs. > > > >> > > > >> > > > > > > > > > > > > > > > > > > > > >
