Re: long waiting times for COPR jobs

[Michal Novotny]

Hello Jean-Marc,

answer is below...

On Tue, Nov 28, 2017 at 11:20 AM, Jean-Marc Liger <
jean-marc.li...@parisdescartes.fr> wrote:

> Hi Clime,
> Le 03/11/2017 à 06:39, Michal Novotny a écrit :
>
> Hey,
>
> On Wed, Nov 1, 2017 at 3:42 PM, Michal Novotny  wrote:
>
>> Hello,
>>
>> lately, COPR pending job queues are holding jobs for pretty long time
>> (even hours). This is a buggy behaviour and we will be doing our best to
>> fix this issue in the following days.
>>
>
> as we have found out yesterday. One of the main issue was
> 'createrepo.lock.lock' file in @rubygems/rubygems/srpm-builds/ directory
> on copr-backend that caused 'createrepo' command to hand indefinitely after
> a srpm build and hence the builder that was allocated for rubygems srpm
> builds was never released afterwards. Together with #160 SRPM build may
> be allocated on multiple builders at once
> , this probably was causing the
> really long waiting times. Our queueing mechaism will still need to be (and
> will be) improved but this was the main issue very likely.
>
> Sorry for the waiting
> COPR team
>
>
> All ppc64le plateforms are in pending mode since yesterday, is it relative
> to the above ?
>

No, not really. This is an OpenStack problem as we are not able to spawn
any new ppc64le builder from COPR. We will try to build a new ppc64le image
based on Fedora27 and see if it helps and if not, we will give another try
to solve it together with OpenStack folks.

clime


> Regards,
> Jean-Marc
>
___
copr-devel mailing list -- copr-devel@lists.fedorahosted.org
To unsubscribe send an email to copr-devel-le...@lists.fedorahosted.org

Re: long waiting times for COPR jobs

[Jean-Marc Liger]

Hi Clime,

Le 03/11/2017 à 06:39, Michal Novotny a écrit :

Hey,

On Wed, Nov 1, 2017 at 3:42 PM, Michal Novotny > wrote:


Hello,

lately, COPR pending job queues are holding jobs for pretty long
time (even hours). This is a buggy behaviour and we will be doing
our best to fix this issue in the following days.


as we have found out yesterday. One of the main issue was 
'createrepo.lock.lock' file in @rubygems/rubygems/srpm-builds/ 
directory on copr-backend that caused 'createrepo' command to hand 
indefinitely after a srpm build and hence the builder that was 
allocated for rubygems srpm builds was never released afterwards. 
Together with #160 SRPM build may be allocated on multiple builders at 
once , this probably was 
causing the really long waiting times. Our queueing mechaism will 
still need to be (and will be) improved but this was the main issue 
very likely.


Sorry for the waiting
COPR team


All ppc64le plateforms are in pending mode since yesterday, is it 
relative to the above ?


Regards,
Jean-Marc
___
copr-devel mailing list -- copr-devel@lists.fedorahosted.org
To unsubscribe send an email to copr-devel-le...@lists.fedorahosted.org

Re: long waiting times for COPR jobs

[Michal Novotny]

Hey,

On Wed, Nov 1, 2017 at 3:42 PM, Michal Novotny  wrote:

> Hello,
>
> lately, COPR pending job queues are holding jobs for pretty long time
> (even hours). This is a buggy behaviour and we will be doing our best to
> fix this issue in the following days.
>

as we have found out yesterday. One of the main issue was
'createrepo.lock.lock' file in @rubygems/rubygems/srpm-builds/ directory on
copr-backend that caused 'createrepo' command to hand indefinitely after a
srpm build and hence the builder that was allocated for rubygems srpm
builds was never released afterwards. Together with #160 SRPM build may be
allocated on multiple builders at once
, this probably was causing the
really long waiting times. Our queueing mechaism will still need to be (and
will be) improved but this was the main issue very likely.

Sorry for the waiting
COPR team
___
copr-devel mailing list -- copr-devel@lists.fedorahosted.org
To unsubscribe send an email to copr-devel-le...@lists.fedorahosted.org

long waiting times for COPR jobs

[Michal Novotny]

Hello,

lately, COPR pending job queues are holding jobs for pretty long time (even
hours). This is a buggy behaviour and we will be doing our best to fix this
issue in the following days.

Thank your for your patience
COPR team
___
copr-devel mailing list -- copr-devel@lists.fedorahosted.org
To unsubscribe send an email to copr-devel-le...@lists.fedorahosted.org