RE: Hadoop Windows Build

[gautham.bangalore]

Hi Ayush,

Thanks for bringing this up. A lot of unit tests are failing on Windows.
Hence, I disabled the unit tests and configured Yetus to run only the
build, so that we catch any regression in this area.

The point that you mentioned about the devs consulting only the report
of Windows build run for merging is concerning. Would it suffice if I add
a row to the report table like this?
"OS: Windows 10 - only build regressions are caught, please consult the
corresponding Linux build as well".

To mitigate this issue, I'll disable the Windows 10 pre-commit pipeline
right away.

Thanks,
--Gautham

-Original Message-
From: Ayush Saxena  
Sent: Tuesday, May 14, 2024 11:58 PM
To: gautham.bangal...@gmail.com
Cc: common-dev@hadoop.apache.org
Subject: Re: Hadoop Windows Build

Hi Gautham,
I think this Windows build has some issues, first is for hadoop-common maven 
site is failing [1] & second it is very confusing for most of the folks, the 
results from Windows build looks same as the normal build, unless you check 
inside that it was a Windows run and it doesn't run tests so I think lot of 
folks are merging based on the results of this windows build, which potentially 
gonna create more issues.

IMHO we should call it off for time being or atleast introduce a way that 
everyone knows that this is the windows build result not the main build result

-Ayush

[1] 
https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-6755/6/artifact/out/patch-mvnsite-root.txt

On Sun, 12 May 2024 at 16:52,  wrote:
>
> Hi folks,
>
> I just wanted to clarify one thing, the nightly build for Windows takes 2 
> days complete.
> The precommit Multibranch pipeline for Windows finishes in about 7 
> hours. I just checked the build queue now, it's just running 2 builds. 
> So, it's not running over all the PRs like before.
>
> I'll try to cache the docker image for the precommit pipeline, it might save 
> us an hour.
>
> Thanks,
> --Gautham
>
> -Original Message-
> From: Allen Wittenauer 
> Sent: Saturday, May 4, 2024 10:20 AM
> To: bui...@apache.org
> Cc: common-dev@hadoop.apache.org
> Subject: Re: Hadoop Windows Build
>
>
>
> > On May 3, 2024, at 9:04 AM, Gavin McDonald  wrote:
> >
> > Build times are in the order of days, not hours, how is the caching 
> > helping here?
>
> It won’t help for full builds but for PRs where it only does 
> parts of the tree it can be dramatic.  (Remember: this is running 
> Yetus which will only rebuild required modules.)
>
>
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>
>
>
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>


-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: Hadoop Windows Build

[Ayush Saxena]

Hi Gautham,
I think this Windows build has some issues, first is for hadoop-common
maven site is failing [1] & second it is very confusing for most of
the folks, the results from Windows build looks same as the normal
build, unless you check inside that it was a Windows run and it
doesn't run tests so I think lot of folks are merging based on the
results of this windows build, which potentially gonna create more
issues.

IMHO we should call it off for time being or atleast introduce a way
that everyone knows that this is the windows build result not the main
build result

-Ayush

[1] 
https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-6755/6/artifact/out/patch-mvnsite-root.txt

On Sun, 12 May 2024 at 16:52,  wrote:
>
> Hi folks,
>
> I just wanted to clarify one thing, the nightly build for Windows takes 2 
> days complete.
> The precommit Multibranch pipeline for Windows finishes in about 7 hours. I 
> just checked
> the build queue now, it's just running 2 builds. So, it's not running over 
> all the PRs like
> before.
>
> I'll try to cache the docker image for the precommit pipeline, it might save 
> us an hour.
>
> Thanks,
> --Gautham
>
> -Original Message-
> From: Allen Wittenauer 
> Sent: Saturday, May 4, 2024 10:20 AM
> To: bui...@apache.org
> Cc: common-dev@hadoop.apache.org
> Subject: Re: Hadoop Windows Build
>
>
>
> > On May 3, 2024, at 9:04 AM, Gavin McDonald  wrote:
> >
> > Build times are in the order of days, not hours, how is the caching
> > helping here?
>
> It won’t help for full builds but for PRs where it only does parts of 
> the tree it can be dramatic.  (Remember: this is running Yetus which will 
> only rebuild required modules.)
>
>
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>
>
>
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

RE: Hadoop Windows Build

[gautham.bangalore]

Hi folks,

I just wanted to clarify one thing, the nightly build for Windows takes 2 days 
complete.
The precommit Multibranch pipeline for Windows finishes in about 7 hours. I 
just checked
the build queue now, it's just running 2 builds. So, it's not running over all 
the PRs like
before.

I'll try to cache the docker image for the precommit pipeline, it might save us 
an hour.

Thanks,
--Gautham

-Original Message-
From: Allen Wittenauer  
Sent: Saturday, May 4, 2024 10:20 AM
To: bui...@apache.org
Cc: common-dev@hadoop.apache.org
Subject: Re: Hadoop Windows Build



> On May 3, 2024, at 9:04 AM, Gavin McDonald  wrote:
> 
> Build times are in the order of days, not hours, how is the caching 
> helping here?

It won’t help for full builds but for PRs where it only does parts of 
the tree it can be dramatic.  (Remember: this is running Yetus which will only 
rebuild required modules.)


-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: Hadoop Windows Build

[Gautham Banasandra]

I get a "gaurava is missing the Overall/Administer permission"
error message upon opening the link. @Chris would you please
run this script once I enable the pipeline? I'll ping you on this thread/
Slack once I enable it.

Thanks,
--Gautham

On Tue, 30 Apr, 2024, 7:07 pm Chris Thistlethwaite, 
wrote:

> I believe this is only available to admins, as there is very high chance
> of wrecking the entire instance running straight groovy on each controller.
> You can try https://ci-hadoop.apache.org/script and see if you have
> access.
>
> -Chris T.
> #asfinfra
>
> On 2024/04/30 09:25:29 gautham.bangal...@gmail.com wrote:
> > Hi Chris,
> >
> > Would it be possible for us (members outside the infra team) to run the
> groovy script to cancel all
> > the jobs? We're still experimenting with the Jenkins configuration so
> that it doesn't run for all the
> > open PRs. So, it would be great if we could run it by ourselves instead
> of reaching out to you folks.
> >
> > Thanks,
> > --Gautham
> >
> > -Original Message-
> > From: Ayush Saxena 
> > Sent: Monday, April 29, 2024 7:29 PM
> > To: Chris Thistlethwaite 
> > Cc: common-dev@hadoop.apache.org
> > Subject: Re: Hadoop Windows Build
> >
> > Thanx Chris, that would be great
> >
> > -Ayush
> >
> > On Mon, 29 Apr 2024 at 19:07, Chris Thistlethwaite 
> > wrote:
> >
> > > I'm following along on lists.a.o. I can cancel all the Windows jobs in
> > > queue, we have a groovy script for that.
> > >
> > > -Chris T.
> > > #asfinfra
> > >
> > > On 2024/04/28 17:35:21 Gautham Banasandra wrote:
> > > > Yeah, I just noticed that. May I know how I can abort all the jobs
> > > > at
> > > once? I only saw that I
> > > > can cancel the jobs one-by-one.
> > > >
> > > > Thanks,
> > > > --Gautham
> > > >
> > > > On 2024/04/28 15:19:13 Ayush Saxena wrote:
> > > > > Thanx Gautham for chasing this.
> > > > >
> > > > > I think there are still some 119 in the build queue, if you see on
> > > > > the
> > > left
> > > > > here [1](Search for Build Queue). They are all stuck on "Waiting
> > > > > for
> > > next
> > > > > available executor on Windows"
> > > > >
> > > > > If you aborted all previously & they showed up now again, then
> > > something is
> > > > > still messed up with the configurations that the pipeline is
> > > > > getting triggered for the existing PR (not new), if you didn't
> > > > > abort earlier
> > > then
> > > > > maybe you need to abort all the ones in queue and free up the
> > > resources.
> > > > >
> > > > > One example of build waiting (as of now) for resource since past 7
> > > hours [2]
> > > > >
> > > > > Let me know if you are stuck, we can together get things figured
> > > > > out
> > > :-)
> > > > >
> > > > > -Ayush
> > > > >
> > > > >
> > > > > [1]
> > > > >
> > > https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/view/ch
> > > ange-requests/builds
> > > > > [2]
> > > > >
> > > https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/view/ch
> > > ange-requests/job/PR-6423/2/console
> > > > >
> > > > > On Sun, 28 Apr 2024 at 13:43, Gautham Banasandra
> > > > > 
> > > wrote:
> > > > >
> > > > > > Hi folks,
> > > > > >
> > > > > > I apologize for the inconvenience caused. I've now applied the
> > > mitigation
> > > > > > described in [3].
> > > > > >
> > > > > > Unfortunately, there are only 12 Windows nodes in the whole
> > > > > > swarm of Jenkins build nodes.
> > > > > > Thus, this caused a starvation of the Windows nodes for other
> > > projects.
> > > > > >
> > > > > > I had reached out to the infra team several months ago and
> > > > > > requested
> > > them
> > > > > > to add more
> > > > > > Windows nodes, but it was turned down. I'm not sure if there's a
> > > > > > way around this, other than getting more Windows nodes.
> > > > > &

Re: Hadoop Windows Build

[Allen Wittenauer]

> On May 3, 2024, at 9:04 AM, Gavin McDonald  wrote:
> 
> Build times are in the order of days, not hours, how is the caching helping
> here?

It won’t help for full builds but for PRs where it only does parts of 
the tree it can be dramatic.  (Remember: this is running Yetus which will only 
rebuild required modules.)


-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: Hadoop Windows Build

[Gavin McDonald]

On Fri, May 3, 2024 at 5:56 PM Allen Wittenauer  wrote:

>
>
> > On Apr 26, 2024, at 9:42 AM, Cesar Hernandez 
> wrote:
> >
> > My two cents is to use cleanWs() instead of deleteDir() as
> > documented in: https://plugins.jenkins.io/ws-cleanup/
>
>
> If this was a generic, run of the mill build, that could be an
> option. Definitely don’t want to do that for Hadoop builds. There is a
> bunch of caching happening to speed things up.  Deleting them would be
> _very_ detrimental to build times.
>

Build times are in the order of days, not hours, how is the caching helping
here?



-- 


*Gavin McDonald - *
Systems Administrator, ASF Infrastructure Team
V.P Travel Assistance Committee

https://tac.apache.org - Applications now open for Community Over Asia 2024
in Hangzhou, China. Don't delay, apply today!

Re: Hadoop Windows Build

[Allen Wittenauer]

> On Apr 26, 2024, at 9:42 AM, Cesar Hernandez  wrote:
> 
> My two cents is to use cleanWs() instead of deleteDir() as
> documented in: https://plugins.jenkins.io/ws-cleanup/


If this was a generic, run of the mill build, that could be an option. 
Definitely don’t want to do that for Hadoop builds. There is a bunch of caching 
happening to speed things up.  Deleting them would be _very_ detrimental to 
build times.




-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

RE: Hadoop Windows Build

[Chris Thistlethwaite]

I believe this is only available to admins, as there is very high chance of 
wrecking the entire instance running straight groovy on each controller. You 
can try https://ci-hadoop.apache.org/script and see if you have access. 

-Chris T.
#asfinfra

On 2024/04/30 09:25:29 gautham.bangal...@gmail.com wrote:
> Hi Chris,
> 
> Would it be possible for us (members outside the infra team) to run the 
> groovy script to cancel all
> the jobs? We're still experimenting with the Jenkins configuration so that it 
> doesn't run for all the
> open PRs. So, it would be great if we could run it by ourselves instead of 
> reaching out to you folks.
> 
> Thanks,
> --Gautham
> 
> -Original Message-
> From: Ayush Saxena  
> Sent: Monday, April 29, 2024 7:29 PM
> To: Chris Thistlethwaite 
> Cc: common-dev@hadoop.apache.org
> Subject: Re: Hadoop Windows Build
> 
> Thanx Chris, that would be great
> 
> -Ayush
> 
> On Mon, 29 Apr 2024 at 19:07, Chris Thistlethwaite 
> wrote:
> 
> > I'm following along on lists.a.o. I can cancel all the Windows jobs in 
> > queue, we have a groovy script for that.
> >
> > -Chris T.
> > #asfinfra
> >
> > On 2024/04/28 17:35:21 Gautham Banasandra wrote:
> > > Yeah, I just noticed that. May I know how I can abort all the jobs 
> > > at
> > once? I only saw that I
> > > can cancel the jobs one-by-one.
> > >
> > > Thanks,
> > > --Gautham
> > >
> > > On 2024/04/28 15:19:13 Ayush Saxena wrote:
> > > > Thanx Gautham for chasing this.
> > > >
> > > > I think there are still some 119 in the build queue, if you see on 
> > > > the
> > left
> > > > here [1](Search for Build Queue). They are all stuck on "Waiting 
> > > > for
> > next
> > > > available executor on Windows"
> > > >
> > > > If you aborted all previously & they showed up now again, then
> > something is
> > > > still messed up with the configurations that the pipeline is 
> > > > getting triggered for the existing PR (not new), if you didn't 
> > > > abort earlier
> > then
> > > > maybe you need to abort all the ones in queue and free up the
> > resources.
> > > >
> > > > One example of build waiting (as of now) for resource since past 7
> > hours [2]
> > > >
> > > > Let me know if you are stuck, we can together get things figured 
> > > > out
> > :-)
> > > >
> > > > -Ayush
> > > >
> > > >
> > > > [1]
> > > >
> > https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/view/ch
> > ange-requests/builds
> > > > [2]
> > > >
> > https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/view/ch
> > ange-requests/job/PR-6423/2/console
> > > >
> > > > On Sun, 28 Apr 2024 at 13:43, Gautham Banasandra 
> > > > 
> > wrote:
> > > >
> > > > > Hi folks,
> > > > >
> > > > > I apologize for the inconvenience caused. I've now applied the
> > mitigation
> > > > > described in [3].
> > > > >
> > > > > Unfortunately, there are only 12 Windows nodes in the whole 
> > > > > swarm of Jenkins build nodes.
> > > > > Thus, this caused a starvation of the Windows nodes for other
> > projects.
> > > > >
> > > > > I had reached out to the infra team several months ago and 
> > > > > requested
> > them
> > > > > to add more
> > > > > Windows nodes, but it was turned down. I'm not sure if there's a 
> > > > > way around this, other than getting more Windows nodes.
> > > > >
> > > > > Thanks,
> > > > > --Gautham
> > > > >
> > > > > On 2024/04/28 04:53:32 Ayush Saxena wrote:
> > > > > > Found this on dev@hadoop -> Moving to common-dev (the ML we 
> > > > > > use)
> > > > > >
> > > > > > I think there was some initiative to enable Windows Pre-Commit 
> > > > > > for
> > every
> > > > > PR
> > > > > > and that seems to have gone wild, either the number of PRs 
> > > > > > raised
> > are way
> > > > > > more than the capacity the nodes can handle or something got
> > > > > misconfigured
> > > > > > in

RE: Hadoop Windows Build

[gautham.bangalore]

Hi Chris,

Would it be possible for us (members outside the infra team) to run the groovy 
script to cancel all
the jobs? We're still experimenting with the Jenkins configuration so that it 
doesn't run for all the
open PRs. So, it would be great if we could run it by ourselves instead of 
reaching out to you folks.

Thanks,
--Gautham

-Original Message-
From: Ayush Saxena  
Sent: Monday, April 29, 2024 7:29 PM
To: Chris Thistlethwaite 
Cc: common-dev@hadoop.apache.org
Subject: Re: Hadoop Windows Build

Thanx Chris, that would be great

-Ayush

On Mon, 29 Apr 2024 at 19:07, Chris Thistlethwaite 
wrote:

> I'm following along on lists.a.o. I can cancel all the Windows jobs in 
> queue, we have a groovy script for that.
>
> -Chris T.
> #asfinfra
>
> On 2024/04/28 17:35:21 Gautham Banasandra wrote:
> > Yeah, I just noticed that. May I know how I can abort all the jobs 
> > at
> once? I only saw that I
> > can cancel the jobs one-by-one.
> >
> > Thanks,
> > --Gautham
> >
> > On 2024/04/28 15:19:13 Ayush Saxena wrote:
> > > Thanx Gautham for chasing this.
> > >
> > > I think there are still some 119 in the build queue, if you see on 
> > > the
> left
> > > here [1](Search for Build Queue). They are all stuck on "Waiting 
> > > for
> next
> > > available executor on Windows"
> > >
> > > If you aborted all previously & they showed up now again, then
> something is
> > > still messed up with the configurations that the pipeline is 
> > > getting triggered for the existing PR (not new), if you didn't 
> > > abort earlier
> then
> > > maybe you need to abort all the ones in queue and free up the
> resources.
> > >
> > > One example of build waiting (as of now) for resource since past 7
> hours [2]
> > >
> > > Let me know if you are stuck, we can together get things figured 
> > > out
> :-)
> > >
> > > -Ayush
> > >
> > >
> > > [1]
> > >
> https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/view/ch
> ange-requests/builds
> > > [2]
> > >
> https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/view/ch
> ange-requests/job/PR-6423/2/console
> > >
> > > On Sun, 28 Apr 2024 at 13:43, Gautham Banasandra 
> > > 
> wrote:
> > >
> > > > Hi folks,
> > > >
> > > > I apologize for the inconvenience caused. I've now applied the
> mitigation
> > > > described in [3].
> > > >
> > > > Unfortunately, there are only 12 Windows nodes in the whole 
> > > > swarm of Jenkins build nodes.
> > > > Thus, this caused a starvation of the Windows nodes for other
> projects.
> > > >
> > > > I had reached out to the infra team several months ago and 
> > > > requested
> them
> > > > to add more
> > > > Windows nodes, but it was turned down. I'm not sure if there's a 
> > > > way around this, other than getting more Windows nodes.
> > > >
> > > > Thanks,
> > > > --Gautham
> > > >
> > > > On 2024/04/28 04:53:32 Ayush Saxena wrote:
> > > > > Found this on dev@hadoop -> Moving to common-dev (the ML we 
> > > > > use)
> > > > >
> > > > > I think there was some initiative to enable Windows Pre-Commit 
> > > > > for
> every
> > > > PR
> > > > > and that seems to have gone wild, either the number of PRs 
> > > > > raised
> are way
> > > > > more than the capacity the nodes can handle or something got
> > > > misconfigured
> > > > > in the job itself that the build is getting triggered for all 
> > > > > the
> open PR
> > > > > not just new, which is leading to starvation of resources.
> > > > >
> > > > > To the best of my knowledge
> > > > > @Gautham Banasandra  / @Iñigo Goiri <
> > > > elgo...@gmail.com> are
> > > > > chasing the initiative, can you folks help check?
> > > > >
> > > > > There are concerns raised by the Infra team here [1] on 
> > > > > dev@hadoop
> > > > >
> > > > > Most probably something messed up while configuring the 
> > > > > hadoop-multibranch-windows job, it shows some 613 PR scheduled
> [2], I
> > > > think
> > > > > it scheduled for all open ones, something similar happened
> long-lon

Re: Hadoop Windows Build

[Ayush Saxena]

Thanx Chris, that would be great

-Ayush

On Mon, 29 Apr 2024 at 19:07, Chris Thistlethwaite 
wrote:

> I'm following along on lists.a.o. I can cancel all the Windows jobs in
> queue, we have a groovy script for that.
>
> -Chris T.
> #asfinfra
>
> On 2024/04/28 17:35:21 Gautham Banasandra wrote:
> > Yeah, I just noticed that. May I know how I can abort all the jobs at
> once? I only saw that I
> > can cancel the jobs one-by-one.
> >
> > Thanks,
> > --Gautham
> >
> > On 2024/04/28 15:19:13 Ayush Saxena wrote:
> > > Thanx Gautham for chasing this.
> > >
> > > I think there are still some 119 in the build queue, if you see on the
> left
> > > here [1](Search for Build Queue). They are all stuck on "Waiting for
> next
> > > available executor on Windows"
> > >
> > > If you aborted all previously & they showed up now again, then
> something is
> > > still messed up with the configurations that the pipeline is getting
> > > triggered for the existing PR (not new), if you didn't abort earlier
> then
> > > maybe you need to abort all the ones in queue and free up the
> resources.
> > >
> > > One example of build waiting (as of now) for resource since past 7
> hours [2]
> > >
> > > Let me know if you are stuck, we can together get things figured out
> :-)
> > >
> > > -Ayush
> > >
> > >
> > > [1]
> > >
> https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/view/change-requests/builds
> > > [2]
> > >
> https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/view/change-requests/job/PR-6423/2/console
> > >
> > > On Sun, 28 Apr 2024 at 13:43, Gautham Banasandra 
> wrote:
> > >
> > > > Hi folks,
> > > >
> > > > I apologize for the inconvenience caused. I've now applied the
> mitigation
> > > > described in [3].
> > > >
> > > > Unfortunately, there are only 12 Windows nodes in the whole swarm of
> > > > Jenkins build nodes.
> > > > Thus, this caused a starvation of the Windows nodes for other
> projects.
> > > >
> > > > I had reached out to the infra team several months ago and requested
> them
> > > > to add more
> > > > Windows nodes, but it was turned down. I'm not sure if there's a way
> > > > around this, other than
> > > > getting more Windows nodes.
> > > >
> > > > Thanks,
> > > > --Gautham
> > > >
> > > > On 2024/04/28 04:53:32 Ayush Saxena wrote:
> > > > > Found this on dev@hadoop -> Moving to common-dev (the ML we use)
> > > > >
> > > > > I think there was some initiative to enable Windows Pre-Commit for
> every
> > > > PR
> > > > > and that seems to have gone wild, either the number of PRs raised
> are way
> > > > > more than the capacity the nodes can handle or something got
> > > > misconfigured
> > > > > in the job itself that the build is getting triggered for all the
> open PR
> > > > > not just new, which is leading to starvation of resources.
> > > > >
> > > > > To the best of my knowledge
> > > > > @Gautham Banasandra  / @Iñigo Goiri <
> > > > elgo...@gmail.com> are
> > > > > chasing the initiative, can you folks help check?
> > > > >
> > > > > There are concerns raised by the Infra team here [1] on dev@hadoop
> > > > >
> > > > > Most probably something messed up while configuring the
> > > > > hadoop-multibranch-windows job, it shows some 613 PR scheduled
> [2], I
> > > > think
> > > > > it scheduled for all open ones, something similar happened
> long-long ago
> > > > > when we were doing migrations, can fetch pointers from [3]
> > > > >
> > > > > [1]
> https://lists.apache.org/thread/7nsyd0vtpb87fhm0fpv8frh6dzk3b3tl
> > > > > [2]
> > > > >
> > > >
> https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/view/change-requests/builds
> > > > > [3]
> https://lists.apache.org/thread/8pxf2yon3r9g61zgv9cf120qnhrs8q23
> > > > >
> > > > > -Ayush
> > > > >
> > > > >
> > > > > On 2024/04/26 16:59:04 Wei-Chiu Chuang wrote:
> > > > > > I'm not familiar with Windows build. But you may have better luck
> > > > reaching
> > > > > > out to Apache Infra
> > > > > > https://infra.apache.org/contact.html
> > > > > >
> > > > > > mailing list, jira or even slack
> > > > > >
> > > > > > On Fri, Apr 26, 2024 at 9:42 AM Cesar Hernandez <
> cesargu...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hello,
> > > > > > > An option that can be implemented in the Hadoop pipeline [1]
> is to
> > > > set a
> > > > > > > timeout [2] on critical stages within the pipelines, for
> example in
> > > > > > > "Windows 10" stage .
> > > > > > > As for the issue the Ci build is logging [3] in the
> > > > hadoop-multibranch
> > > > > jobs
> > > > > > > reported by Chris, it seems the issue is around the Post
> (cleanup)
> > > > > pipeline
> > > > > > > process. My two cents is to use cleanWs() instead of
> deleteDir() as
> > > > > > > documented in: https://plugins.jenkins.io/ws-cleanup/
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > > > >
> > > > >
> > > >
> https://github.com/apache/hadoop/blob/trunk/dev-support/jenkinsfile-windows-10
> > > > > > >
> > > > > > > [2]
> > > > > > >
> > > > > > >
> > 

Re: Hadoop Windows Build

[Chris Thistlethwaite]

I'm following along on lists.a.o. I can cancel all the Windows jobs in queue, 
we have a groovy script for that.

-Chris T.
#asfinfra

On 2024/04/28 17:35:21 Gautham Banasandra wrote:
> Yeah, I just noticed that. May I know how I can abort all the jobs at once? I 
> only saw that I
> can cancel the jobs one-by-one.
> 
> Thanks,
> --Gautham
> 
> On 2024/04/28 15:19:13 Ayush Saxena wrote:
> > Thanx Gautham for chasing this.
> > 
> > I think there are still some 119 in the build queue, if you see on the left
> > here [1](Search for Build Queue). They are all stuck on "Waiting for next
> > available executor on Windows"
> > 
> > If you aborted all previously & they showed up now again, then something is
> > still messed up with the configurations that the pipeline is getting
> > triggered for the existing PR (not new), if you didn't abort earlier then
> > maybe you need to abort all the ones in queue and free up the resources.
> > 
> > One example of build waiting (as of now) for resource since past 7 hours [2]
> > 
> > Let me know if you are stuck, we can together get things figured out :-)
> > 
> > -Ayush
> > 
> > 
> > [1]
> > https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/view/change-requests/builds
> > [2]
> > https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/view/change-requests/job/PR-6423/2/console
> > 
> > On Sun, 28 Apr 2024 at 13:43, Gautham Banasandra  wrote:
> > 
> > > Hi folks,
> > >
> > > I apologize for the inconvenience caused. I've now applied the mitigation
> > > described in [3].
> > >
> > > Unfortunately, there are only 12 Windows nodes in the whole swarm of
> > > Jenkins build nodes.
> > > Thus, this caused a starvation of the Windows nodes for other projects.
> > >
> > > I had reached out to the infra team several months ago and requested them
> > > to add more
> > > Windows nodes, but it was turned down. I'm not sure if there's a way
> > > around this, other than
> > > getting more Windows nodes.
> > >
> > > Thanks,
> > > --Gautham
> > >
> > > On 2024/04/28 04:53:32 Ayush Saxena wrote:
> > > > Found this on dev@hadoop -> Moving to common-dev (the ML we use)
> > > >
> > > > I think there was some initiative to enable Windows Pre-Commit for every
> > > PR
> > > > and that seems to have gone wild, either the number of PRs raised are 
> > > > way
> > > > more than the capacity the nodes can handle or something got
> > > misconfigured
> > > > in the job itself that the build is getting triggered for all the open 
> > > > PR
> > > > not just new, which is leading to starvation of resources.
> > > >
> > > > To the best of my knowledge
> > > > @Gautham Banasandra  / @Iñigo Goiri <
> > > elgo...@gmail.com> are
> > > > chasing the initiative, can you folks help check?
> > > >
> > > > There are concerns raised by the Infra team here [1] on dev@hadoop
> > > >
> > > > Most probably something messed up while configuring the
> > > > hadoop-multibranch-windows job, it shows some 613 PR scheduled [2], I
> > > think
> > > > it scheduled for all open ones, something similar happened long-long ago
> > > > when we were doing migrations, can fetch pointers from [3]
> > > >
> > > > [1] https://lists.apache.org/thread/7nsyd0vtpb87fhm0fpv8frh6dzk3b3tl
> > > > [2]
> > > >
> > > https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/view/change-requests/builds
> > > > [3] https://lists.apache.org/thread/8pxf2yon3r9g61zgv9cf120qnhrs8q23
> > > >
> > > > -Ayush
> > > >
> > > >
> > > > On 2024/04/26 16:59:04 Wei-Chiu Chuang wrote:
> > > > > I'm not familiar with Windows build. But you may have better luck
> > > reaching
> > > > > out to Apache Infra
> > > > > https://infra.apache.org/contact.html
> > > > >
> > > > > mailing list, jira or even slack
> > > > >
> > > > > On Fri, Apr 26, 2024 at 9:42 AM Cesar Hernandez 
> > > > > wrote:
> > > > >
> > > > > > Hello,
> > > > > > An option that can be implemented in the Hadoop pipeline [1] is to
> > > set a
> > > > > > timeout [2] on critical stages within the pipelines, for example in
> > > > > > "Windows 10" stage .
> > > > > > As for the issue the Ci build is logging [3] in the
> > > hadoop-multibranch
> > > > jobs
> > > > > > reported by Chris, it seems the issue is around the Post (cleanup)
> > > > pipeline
> > > > > > process. My two cents is to use cleanWs() instead of deleteDir() as
> > > > > > documented in: https://plugins.jenkins.io/ws-cleanup/
> > > > > >
> > > > > > [1]
> > > > > >
> > > > > >
> > > >
> > > https://github.com/apache/hadoop/blob/trunk/dev-support/jenkinsfile-windows-10
> > > > > >
> > > > > > [2]
> > > > > >
> > > > > >
> > > >
> > > https://www.jenkins.io/doc/pipeline/steps/workflow-basic-steps/#timeout-enforce-time-limit
> > > > > >
> > > > > > [3]
> > > > > >
> > > > > > Still waiting to schedule task
> > > > > > Waiting for next available executor on ‘Windows
> > > > > > ’[Pipeline] //
> > > > > > node[Pipeline] stage
> > > > > > <
> > > > > >
> > > >
> > > 

Re: Hadoop Windows Build

[Gautham Banasandra]

Yeah, I just noticed that. May I know how I can abort all the jobs at once? I 
only saw that I
can cancel the jobs one-by-one.

Thanks,
--Gautham

On 2024/04/28 15:19:13 Ayush Saxena wrote:
> Thanx Gautham for chasing this.
> 
> I think there are still some 119 in the build queue, if you see on the left
> here [1](Search for Build Queue). They are all stuck on "Waiting for next
> available executor on Windows"
> 
> If you aborted all previously & they showed up now again, then something is
> still messed up with the configurations that the pipeline is getting
> triggered for the existing PR (not new), if you didn't abort earlier then
> maybe you need to abort all the ones in queue and free up the resources.
> 
> One example of build waiting (as of now) for resource since past 7 hours [2]
> 
> Let me know if you are stuck, we can together get things figured out :-)
> 
> -Ayush
> 
> 
> [1]
> https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/view/change-requests/builds
> [2]
> https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/view/change-requests/job/PR-6423/2/console
> 
> On Sun, 28 Apr 2024 at 13:43, Gautham Banasandra  wrote:
> 
> > Hi folks,
> >
> > I apologize for the inconvenience caused. I've now applied the mitigation
> > described in [3].
> >
> > Unfortunately, there are only 12 Windows nodes in the whole swarm of
> > Jenkins build nodes.
> > Thus, this caused a starvation of the Windows nodes for other projects.
> >
> > I had reached out to the infra team several months ago and requested them
> > to add more
> > Windows nodes, but it was turned down. I'm not sure if there's a way
> > around this, other than
> > getting more Windows nodes.
> >
> > Thanks,
> > --Gautham
> >
> > On 2024/04/28 04:53:32 Ayush Saxena wrote:
> > > Found this on dev@hadoop -> Moving to common-dev (the ML we use)
> > >
> > > I think there was some initiative to enable Windows Pre-Commit for every
> > PR
> > > and that seems to have gone wild, either the number of PRs raised are way
> > > more than the capacity the nodes can handle or something got
> > misconfigured
> > > in the job itself that the build is getting triggered for all the open PR
> > > not just new, which is leading to starvation of resources.
> > >
> > > To the best of my knowledge
> > > @Gautham Banasandra  / @Iñigo Goiri <
> > elgo...@gmail.com> are
> > > chasing the initiative, can you folks help check?
> > >
> > > There are concerns raised by the Infra team here [1] on dev@hadoop
> > >
> > > Most probably something messed up while configuring the
> > > hadoop-multibranch-windows job, it shows some 613 PR scheduled [2], I
> > think
> > > it scheduled for all open ones, something similar happened long-long ago
> > > when we were doing migrations, can fetch pointers from [3]
> > >
> > > [1] https://lists.apache.org/thread/7nsyd0vtpb87fhm0fpv8frh6dzk3b3tl
> > > [2]
> > >
> > https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/view/change-requests/builds
> > > [3] https://lists.apache.org/thread/8pxf2yon3r9g61zgv9cf120qnhrs8q23
> > >
> > > -Ayush
> > >
> > >
> > > On 2024/04/26 16:59:04 Wei-Chiu Chuang wrote:
> > > > I'm not familiar with Windows build. But you may have better luck
> > reaching
> > > > out to Apache Infra
> > > > https://infra.apache.org/contact.html
> > > >
> > > > mailing list, jira or even slack
> > > >
> > > > On Fri, Apr 26, 2024 at 9:42 AM Cesar Hernandez 
> > > > wrote:
> > > >
> > > > > Hello,
> > > > > An option that can be implemented in the Hadoop pipeline [1] is to
> > set a
> > > > > timeout [2] on critical stages within the pipelines, for example in
> > > > > "Windows 10" stage .
> > > > > As for the issue the Ci build is logging [3] in the
> > hadoop-multibranch
> > > jobs
> > > > > reported by Chris, it seems the issue is around the Post (cleanup)
> > > pipeline
> > > > > process. My two cents is to use cleanWs() instead of deleteDir() as
> > > > > documented in: https://plugins.jenkins.io/ws-cleanup/
> > > > >
> > > > > [1]
> > > > >
> > > > >
> > >
> > https://github.com/apache/hadoop/blob/trunk/dev-support/jenkinsfile-windows-10
> > > > >
> > > > > [2]
> > > > >
> > > > >
> > >
> > https://www.jenkins.io/doc/pipeline/steps/workflow-basic-steps/#timeout-enforce-time-limit
> > > > >
> > > > > [3]
> > > > >
> > > > > Still waiting to schedule task
> > > > > Waiting for next available executor on ‘Windows
> > > > > ’[Pipeline] //
> > > > > node[Pipeline] stage
> > > > > <
> > > > >
> > >
> > https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-1137/1/console#
> > > > > >[Pipeline]
> > > > > { (Declarative: Post Actions)
> > > > > <
> > > > >
> > >
> > https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-1137/1/console#
> > > > > >[Pipeline]
> > > > > script <
> > > > >
> > >
> > https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-1137/1/console#
> > > > > >[Pipeline]
> > > > > { <
> > > > >
> > >

Re: Hadoop Windows Build

[Ayush Saxena]

Thanx Gautham for chasing this.

I think there are still some 119 in the build queue, if you see on the left
here [1](Search for Build Queue). They are all stuck on "Waiting for next
available executor on Windows"

If you aborted all previously & they showed up now again, then something is
still messed up with the configurations that the pipeline is getting
triggered for the existing PR (not new), if you didn't abort earlier then
maybe you need to abort all the ones in queue and free up the resources.

One example of build waiting (as of now) for resource since past 7 hours [2]

Let me know if you are stuck, we can together get things figured out :-)

-Ayush


[1]
https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/view/change-requests/builds
[2]
https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/view/change-requests/job/PR-6423/2/console

On Sun, 28 Apr 2024 at 13:43, Gautham Banasandra  wrote:

> Hi folks,
>
> I apologize for the inconvenience caused. I've now applied the mitigation
> described in [3].
>
> Unfortunately, there are only 12 Windows nodes in the whole swarm of
> Jenkins build nodes.
> Thus, this caused a starvation of the Windows nodes for other projects.
>
> I had reached out to the infra team several months ago and requested them
> to add more
> Windows nodes, but it was turned down. I'm not sure if there's a way
> around this, other than
> getting more Windows nodes.
>
> Thanks,
> --Gautham
>
> On 2024/04/28 04:53:32 Ayush Saxena wrote:
> > Found this on dev@hadoop -> Moving to common-dev (the ML we use)
> >
> > I think there was some initiative to enable Windows Pre-Commit for every
> PR
> > and that seems to have gone wild, either the number of PRs raised are way
> > more than the capacity the nodes can handle or something got
> misconfigured
> > in the job itself that the build is getting triggered for all the open PR
> > not just new, which is leading to starvation of resources.
> >
> > To the best of my knowledge
> > @Gautham Banasandra  / @Iñigo Goiri <
> elgo...@gmail.com> are
> > chasing the initiative, can you folks help check?
> >
> > There are concerns raised by the Infra team here [1] on dev@hadoop
> >
> > Most probably something messed up while configuring the
> > hadoop-multibranch-windows job, it shows some 613 PR scheduled [2], I
> think
> > it scheduled for all open ones, something similar happened long-long ago
> > when we were doing migrations, can fetch pointers from [3]
> >
> > [1] https://lists.apache.org/thread/7nsyd0vtpb87fhm0fpv8frh6dzk3b3tl
> > [2]
> >
> https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/view/change-requests/builds
> > [3] https://lists.apache.org/thread/8pxf2yon3r9g61zgv9cf120qnhrs8q23
> >
> > -Ayush
> >
> >
> > On 2024/04/26 16:59:04 Wei-Chiu Chuang wrote:
> > > I'm not familiar with Windows build. But you may have better luck
> reaching
> > > out to Apache Infra
> > > https://infra.apache.org/contact.html
> > >
> > > mailing list, jira or even slack
> > >
> > > On Fri, Apr 26, 2024 at 9:42 AM Cesar Hernandez 
> > > wrote:
> > >
> > > > Hello,
> > > > An option that can be implemented in the Hadoop pipeline [1] is to
> set a
> > > > timeout [2] on critical stages within the pipelines, for example in
> > > > "Windows 10" stage .
> > > > As for the issue the Ci build is logging [3] in the
> hadoop-multibranch
> > jobs
> > > > reported by Chris, it seems the issue is around the Post (cleanup)
> > pipeline
> > > > process. My two cents is to use cleanWs() instead of deleteDir() as
> > > > documented in: https://plugins.jenkins.io/ws-cleanup/
> > > >
> > > > [1]
> > > >
> > > >
> >
> https://github.com/apache/hadoop/blob/trunk/dev-support/jenkinsfile-windows-10
> > > >
> > > > [2]
> > > >
> > > >
> >
> https://www.jenkins.io/doc/pipeline/steps/workflow-basic-steps/#timeout-enforce-time-limit
> > > >
> > > > [3]
> > > >
> > > > Still waiting to schedule task
> > > > Waiting for next available executor on ‘Windows
> > > > ’[Pipeline] //
> > > > node[Pipeline] stage
> > > > <
> > > >
> >
> https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-1137/1/console#
> > > > >[Pipeline]
> > > > { (Declarative: Post Actions)
> > > > <
> > > >
> >
> https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-1137/1/console#
> > > > >[Pipeline]
> > > > script <
> > > >
> >
> https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-1137/1/console#
> > > > >[Pipeline]
> > > > { <
> > > >
> >
> https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-1137/1/console#
> > > > >[Pipeline]
> > > > deleteDir <
> > > >
> >
> https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-1137/1/console#
> > > > >[Pipeline]
> > > > }[Pipeline] // scriptError when executing cleanup post condition:
> > > > Also:   org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId:
> > > > ca1b7f2f-ec16-4bde-ac51-85f964794e37
> > > > 

Re: Hadoop Windows Build

[Gautham Banasandra]

Hi folks,

I apologize for the inconvenience caused. I've now applied the mitigation 
described in [3].

Unfortunately, there are only 12 Windows nodes in the whole swarm of Jenkins 
build nodes.
Thus, this caused a starvation of the Windows nodes for other projects.

I had reached out to the infra team several months ago and requested them to 
add more
Windows nodes, but it was turned down. I'm not sure if there's a way around 
this, other than
getting more Windows nodes.

Thanks,
--Gautham

On 2024/04/28 04:53:32 Ayush Saxena wrote:
> Found this on dev@hadoop -> Moving to common-dev (the ML we use)
> 
> I think there was some initiative to enable Windows Pre-Commit for every PR
> and that seems to have gone wild, either the number of PRs raised are way
> more than the capacity the nodes can handle or something got misconfigured
> in the job itself that the build is getting triggered for all the open PR
> not just new, which is leading to starvation of resources.
> 
> To the best of my knowledge
> @Gautham Banasandra  / @Iñigo Goiri  
> are
> chasing the initiative, can you folks help check?
> 
> There are concerns raised by the Infra team here [1] on dev@hadoop
> 
> Most probably something messed up while configuring the
> hadoop-multibranch-windows job, it shows some 613 PR scheduled [2], I think
> it scheduled for all open ones, something similar happened long-long ago
> when we were doing migrations, can fetch pointers from [3]
> 
> [1] https://lists.apache.org/thread/7nsyd0vtpb87fhm0fpv8frh6dzk3b3tl
> [2]
> https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/view/change-requests/builds
> [3] https://lists.apache.org/thread/8pxf2yon3r9g61zgv9cf120qnhrs8q23
> 
> -Ayush
> 
> 
> On 2024/04/26 16:59:04 Wei-Chiu Chuang wrote:
> > I'm not familiar with Windows build. But you may have better luck reaching
> > out to Apache Infra
> > https://infra.apache.org/contact.html
> >
> > mailing list, jira or even slack
> >
> > On Fri, Apr 26, 2024 at 9:42 AM Cesar Hernandez 
> > wrote:
> >
> > > Hello,
> > > An option that can be implemented in the Hadoop pipeline [1] is to set a
> > > timeout [2] on critical stages within the pipelines, for example in
> > > "Windows 10" stage .
> > > As for the issue the Ci build is logging [3] in the hadoop-multibranch
> jobs
> > > reported by Chris, it seems the issue is around the Post (cleanup)
> pipeline
> > > process. My two cents is to use cleanWs() instead of deleteDir() as
> > > documented in: https://plugins.jenkins.io/ws-cleanup/
> > >
> > > [1]
> > >
> > >
> https://github.com/apache/hadoop/blob/trunk/dev-support/jenkinsfile-windows-10
> > >
> > > [2]
> > >
> > >
> https://www.jenkins.io/doc/pipeline/steps/workflow-basic-steps/#timeout-enforce-time-limit
> > >
> > > [3]
> > >
> > > Still waiting to schedule task
> > > Waiting for next available executor on ‘Windows
> > > ’[Pipeline] //
> > > node[Pipeline] stage
> > > <
> > >
> https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-1137/1/console#
> > > >[Pipeline]
> > > { (Declarative: Post Actions)
> > > <
> > >
> https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-1137/1/console#
> > > >[Pipeline]
> > > script <
> > >
> https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-1137/1/console#
> > > >[Pipeline]
> > > { <
> > >
> https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-1137/1/console#
> > > >[Pipeline]
> > > deleteDir <
> > >
> https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-1137/1/console#
> > > >[Pipeline]
> > > }[Pipeline] // scriptError when executing cleanup post condition:
> > > Also:   org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId:
> > > ca1b7f2f-ec16-4bde-ac51-85f964794e37
> > > org.jenkinsci.plugins.workflow.steps.MissingContextVariableException:
> > > Required context class hudson.FilePath is missing
> > > Perhaps you forgot to surround the code with a step that provides
> > > this, such as: node
> > > at
> > >
> org.jenkinsci.plugins.workflow.steps.StepDescriptor.checkContextAvailability(StepDescriptor.java:265)
> > > at
> org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:300)
> > > at
> > > org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:196)
> > > at
> > >
> org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:124)
> > > at
> jdk.internal.reflect.GeneratedMethodAccessor1084.invoke(Unknown
> > > Source)
> > > at
> > >
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> > > at
> > > org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
> > > at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
> > > at
> groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1225)
> > >

Re: Hadoop Windows Build

[Ayush Saxena]

Found this on dev@hadoop -> Moving to common-dev (the ML we use)

I think there was some initiative to enable Windows Pre-Commit for every PR
and that seems to have gone wild, either the number of PRs raised are way
more than the capacity the nodes can handle or something got misconfigured
in the job itself that the build is getting triggered for all the open PR
not just new, which is leading to starvation of resources.

To the best of my knowledge
@Gautham Banasandra  / @Iñigo Goiri  are
chasing the initiative, can you folks help check?

There are concerns raised by the Infra team here [1] on dev@hadoop

Most probably something messed up while configuring the
hadoop-multibranch-windows job, it shows some 613 PR scheduled [2], I think
it scheduled for all open ones, something similar happened long-long ago
when we were doing migrations, can fetch pointers from [3]

[1] https://lists.apache.org/thread/7nsyd0vtpb87fhm0fpv8frh6dzk3b3tl
[2]
https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/view/change-requests/builds
[3] https://lists.apache.org/thread/8pxf2yon3r9g61zgv9cf120qnhrs8q23

-Ayush


On 2024/04/26 16:59:04 Wei-Chiu Chuang wrote:
> I'm not familiar with Windows build. But you may have better luck reaching
> out to Apache Infra
> https://infra.apache.org/contact.html
>
> mailing list, jira or even slack
>
> On Fri, Apr 26, 2024 at 9:42 AM Cesar Hernandez 
> wrote:
>
> > Hello,
> > An option that can be implemented in the Hadoop pipeline [1] is to set a
> > timeout [2] on critical stages within the pipelines, for example in
> > "Windows 10" stage .
> > As for the issue the Ci build is logging [3] in the hadoop-multibranch
jobs
> > reported by Chris, it seems the issue is around the Post (cleanup)
pipeline
> > process. My two cents is to use cleanWs() instead of deleteDir() as
> > documented in: https://plugins.jenkins.io/ws-cleanup/
> >
> > [1]
> >
> >
https://github.com/apache/hadoop/blob/trunk/dev-support/jenkinsfile-windows-10
> >
> > [2]
> >
> >
https://www.jenkins.io/doc/pipeline/steps/workflow-basic-steps/#timeout-enforce-time-limit
> >
> > [3]
> >
> > Still waiting to schedule task
> > Waiting for next available executor on ‘Windows
> > ’[Pipeline] //
> > node[Pipeline] stage
> > <
> >
https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-1137/1/console#
> > >[Pipeline]
> > { (Declarative: Post Actions)
> > <
> >
https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-1137/1/console#
> > >[Pipeline]
> > script <
> >
https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-1137/1/console#
> > >[Pipeline]
> > { <
> >
https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-1137/1/console#
> > >[Pipeline]
> > deleteDir <
> >
https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-1137/1/console#
> > >[Pipeline]
> > }[Pipeline] // scriptError when executing cleanup post condition:
> > Also:   org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId:
> > ca1b7f2f-ec16-4bde-ac51-85f964794e37
> > org.jenkinsci.plugins.workflow.steps.MissingContextVariableException:
> > Required context class hudson.FilePath is missing
> > Perhaps you forgot to surround the code with a step that provides
> > this, such as: node
> > at
> >
org.jenkinsci.plugins.workflow.steps.StepDescriptor.checkContextAvailability(StepDescriptor.java:265)
> > at
org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:300)
> > at
> > org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:196)
> > at
> >
org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:124)
> > at
jdk.internal.reflect.GeneratedMethodAccessor1084.invoke(Unknown
> > Source)
> > at
> >
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> > at
> > org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
> > at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
> > at
groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1225)
> > at
groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
> > at
> >
org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:41)
> > at
> >
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
> > at
> >
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
> > at
org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:180)
> > at
> >
org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
> > at
> >
org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:163)
> > at