Re: Rawhide: where for art thou? (why no rawhide composes recently)

[Adam Williamson]

On Wed, 2017-06-14 at 16:43 +0100, Peter Robinson wrote:
> 
> Yes, I'm well aware of the plans but it doesn't help us now. There was
> over 3500 package updates in the compose, it shows how quickly we got
> behind.

FWIW I really see this as a resource/priority problem, not a process
problem. We just don't have enough people to care about everything all
at once; when there are other emergencies, 'Rawhide didn't compose'
tends to get kicked down the priority list. If we had someone whose
main job was to care about Rawhide, then that person would jump on it
the moment any Rawhide compose failed. Instead what tends to happen is
that multiple problems build up because we don't have enough person-
time to really roll the trucks every time Rawhide fails to compose, so
we wind up with six different things to fix when we eventually do get
to fixing it.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide: where for art thou? (why no rawhide composes recently)

[Martin Kolman]

On Wed, 2017-06-14 at 14:58 +0100, Daniel P. Berrange wrote:
> On Tue, Jun 13, 2017 at 07:11:21AM -0600, Kevin Fenzi wrote:
> > Greetings.
> > 
> > Some folks may have noticed that there have been no completed
> > rawhide
> > composes in a while (13 days as of today).
> > 
> > This has been due to a variety of bugs and issues, along with pungi
> > now
> > failing composes that don't have all required release blocking
> > items.
> > 
> > Here's a partial list:
> > 
> > 2017-06-01 - lorax traceback, bug 1457055
> > 2017-06-02 - another lorax issue, bug 1457906
> > 2017-06-03 - cloud base failed in anaconda, bug 1458509
> > 2017-06-04 - ditto
> > 2017-06-05 - ditto
> > 2017-06-06 - pungi bug - https://pagure.io/pungi/issue/641
> > 2017-06-07 - ditto
> > 2017-06-08 - ditto
> > 2017-06-09 - ditto
> > 2017-06-10 - ditto
> > 2017-06-11 - ditto
> > 2017-06-12 - ditto
> > 2017-06-12.1 - pungi bug fixed, but hit libgtop2 broken deps in
> > metacity
> > that failed the comppose. I fixed those (and control-center) last
> > night.
> > 2017-06-13 - still running, cross your fingers.
> 
> [snip]
> 
> > Anyhow, hopefully we will have a rawhide compose today, and if not
> > I
> > will keep poking it to get it going...
> 
> Unfortunately while the latest compose succeeded, actually trying to
> use the installer resulted in a python traceback pretty much
> immediately
> 
>https://bugzilla.redhat.com/show_bug.cgi?id=1461469
Should be fixed once anaconda-27.13-1 hits a compose.

Best Wishes
Martin Kolman

> 
> Regards,
> Daniel
> -- 
> > : https://berrange.com  
> > -o-https://www.flickr.com/photos/dberrange :|
> > : https://libvirt.org 
> > -o-https://fstop138.berrange.com :|
> > : https://entangle-photo.org
> > -o-https://www.instagram.com/dberrange :|
> 
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide: where for art thou? (why no rawhide composes recently)

[Kevin Fenzi]

On 06/14/2017 10:00 AM, Jonathan Dieter wrote:
> On Wed, 2017-06-14 at 09:36 -0600, Kevin Fenzi wrote:
>> * Composes are really slow (likely related to storage slowness), if they
>> were faster or could fail faster we could untag/fix/iterate more. Right
>> now we are lucky to get 2 chances a day.
> 
> When you're mentioning storage slowness, are we talking a distributed
> filesystem?  If so, is this an issue with metadata lookups being slow
> or is it slow data access?

Nope, it's storage provided by our primary sponsor (Red Hat) via netapp.
I'm working with them to try and track down what the issue is...
starting around may or so things just got slower and we have not yet
been able to tell why. ;(
> The only reason I ask is that we switched over to LizardFS because its
> metadata lookups were *much* faster than what I'd seen from any of the
> alternatives.

Good to know. ;)

kevin




signature.asc
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide: where for art thou? (why no rawhide composes recently)

[Kevin Fenzi]

On 06/14/2017 04:16 AM, Martin Kolman wrote:

> I agree with that - for example the Anaconda installer CI broke due to
> a rawhide package change in the last "working" compose. There is a
> fixed package available and built in Koji, but it's not reaching the
> Rawhide repos due to the broken composes. In this case we can fix the
> CI by using a COPR build of the package, but in other cases it could be
> more difficult.
> 
> BTW, when was the change to only release packages to repositories once
> a compose succeeds introduced ? I kinda always thought packages go to
> the Rawhide repos almost immediately - mirroring machinery permitting.

Rawhide has always been a daily compose of things that built in the last
day.

I'm not sure when the change to fail composes when blocking deliverables
failed landed. I want to say the Beginning of may, but I could be wrong.
Dennis should know more..

kevin



signature.asc
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide: where for art thou? (why no rawhide composes recently)

[Jonathan Dieter]

On Wed, 2017-06-14 at 09:36 -0600, Kevin Fenzi wrote:
> * Composes are really slow (likely related to storage slowness), if they
> were faster or could fail faster we could untag/fix/iterate more. Right
> now we are lucky to get 2 chances a day.

When you're mentioning storage slowness, are we talking a distributed
filesystem?  If so, is this an issue with metadata lookups being slow
or is it slow data access?

The only reason I ask is that we switched over to LizardFS because its
metadata lookups were *much* faster than what I'd seen from any of the
alternatives.

Jonathan

signature.asc
Description: This is a digitally signed message part
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide: where for art thou? (why no rawhide composes recently)

[Peter Robinson]

On Wed, Jun 14, 2017 at 4:36 PM, Kevin Fenzi  wrote:
> On 06/13/2017 03:47 PM, Peter Robinson wrote:
>>
>> For actual artifacts such as cloud/disk/installer images I agree but
>> at least pushing out individual packages so people can do "dnf
>> upgrade" picks up issues such as dependency issues that also kill the
>> compose and allows people to still test explicit parts and have the
>> composite parts of a "rolling release" and get things fixed I feel
>> that's better than dragging everything to a blinding halt like we have
>> for the last 13... or is it 14 days?
>
> Perhaps so. Note that I personally have been updating from the koji
> repo, but I know that has some limitations (no multiarch, etc).
>
>> I can't help but feel this is like British politics is ATM where
>> people are claiming everything is "strong and stable" while the wheels
>> have fallen off and are rolling down the road. I don't think pushing
>> out the Everything repo stops the "kill off Alpha" process from
>> happening, in fact I believe it means it's more likely to happen
>> because if the last two weeks shows anything all that happens is that
>> if we wait for a "everything is perfect ship it" we never will and
>> because nothing is shipped nothing is tested so once we get to the
>> "computer thinks it's good" we can get to the actual testing and then
>> we get to "what the hell changed in the last two weeks broke X, Y and
>> Z, are they related or completely independent?" process.
>>
>> I think the all or nothing actually makes it less likely for us to
>> ever ship anything! I don't think shipping the traditional
>> "Everything" repo breaks the "kill Alpha" proposal, I think we need to
>> be pragmatic and realise that people actually consuming content helps
>> that.
>>
>> Peter "yes I live in the as strong and stable as a house of cards
>> country so I can joke/comment on it" R
>
> Well, I am not even an owner of that Change, so I will leave it to Adam
> and Dennis to chime in on the no alpha part.
>
> I do think now that we do have a compose that worked we are in a better
> place and can now do what Colin was suggesting (If something fails now,
> just untag the offending thing and file a bug on it and recompose).
>
> There's definitely tons of room for improvement all around here including:
>
> * Composes are really slow (likely related to storage slowness), if they
> were faster or could fail faster we could untag/fix/iterate more. Right
> now we are lucky to get 2 chances a day.
>
> * There's plans pretty far along to 'pre test' base stuff with test
> composes, that should help a lot. ie, new anaconda or lorax or whatever
> arrives, a test compose is done and openqa tests it. If it fails we know
> we can't use that in the main compose.

Yes, I'm well aware of the plans but it doesn't help us now. There was
over 3500 package updates in the compose, it shows how quickly we got
behind.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide: where for art thou? (why no rawhide composes recently)

[Kevin Fenzi]

On 06/13/2017 03:47 PM, Peter Robinson wrote:
>
> For actual artifacts such as cloud/disk/installer images I agree but
> at least pushing out individual packages so people can do "dnf
> upgrade" picks up issues such as dependency issues that also kill the
> compose and allows people to still test explicit parts and have the
> composite parts of a "rolling release" and get things fixed I feel
> that's better than dragging everything to a blinding halt like we have
> for the last 13... or is it 14 days?

Perhaps so. Note that I personally have been updating from the koji
repo, but I know that has some limitations (no multiarch, etc).

> I can't help but feel this is like British politics is ATM where
> people are claiming everything is "strong and stable" while the wheels
> have fallen off and are rolling down the road. I don't think pushing
> out the Everything repo stops the "kill off Alpha" process from
> happening, in fact I believe it means it's more likely to happen
> because if the last two weeks shows anything all that happens is that
> if we wait for a "everything is perfect ship it" we never will and
> because nothing is shipped nothing is tested so once we get to the
> "computer thinks it's good" we can get to the actual testing and then
> we get to "what the hell changed in the last two weeks broke X, Y and
> Z, are they related or completely independent?" process.
> 
> I think the all or nothing actually makes it less likely for us to
> ever ship anything! I don't think shipping the traditional
> "Everything" repo breaks the "kill Alpha" proposal, I think we need to
> be pragmatic and realise that people actually consuming content helps
> that.
> 
> Peter "yes I live in the as strong and stable as a house of cards
> country so I can joke/comment on it" R

Well, I am not even an owner of that Change, so I will leave it to Adam
and Dennis to chime in on the no alpha part.

I do think now that we do have a compose that worked we are in a better
place and can now do what Colin was suggesting (If something fails now,
just untag the offending thing and file a bug on it and recompose).

There's definitely tons of room for improvement all around here including:

* Composes are really slow (likely related to storage slowness), if they
were faster or could fail faster we could untag/fix/iterate more. Right
now we are lucky to get 2 chances a day.

* There's plans pretty far along to 'pre test' base stuff with test
composes, that should help a lot. ie, new anaconda or lorax or whatever
arrives, a test compose is done and openqa tests it. If it fails we know
we can't use that in the main compose.

kevin





signature.asc
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide: where for art thou? (why no rawhide composes recently)

[Daniel P. Berrange]

On Tue, Jun 13, 2017 at 07:11:21AM -0600, Kevin Fenzi wrote:
> Greetings.
> 
> Some folks may have noticed that there have been no completed rawhide
> composes in a while (13 days as of today).
> 
> This has been due to a variety of bugs and issues, along with pungi now
> failing composes that don't have all required release blocking items.
> 
> Here's a partial list:
> 
> 2017-06-01 - lorax traceback, bug 1457055
> 2017-06-02 - another lorax issue, bug 1457906
> 2017-06-03 - cloud base failed in anaconda, bug 1458509
> 2017-06-04 - ditto
> 2017-06-05 - ditto
> 2017-06-06 - pungi bug - https://pagure.io/pungi/issue/641
> 2017-06-07 - ditto
> 2017-06-08 - ditto
> 2017-06-09 - ditto
> 2017-06-10 - ditto
> 2017-06-11 - ditto
> 2017-06-12 - ditto
> 2017-06-12.1 - pungi bug fixed, but hit libgtop2 broken deps in metacity
> that failed the comppose. I fixed those (and control-center) last night.
> 2017-06-13 - still running, cross your fingers.

[snip]

> Anyhow, hopefully we will have a rawhide compose today, and if not I
> will keep poking it to get it going...

Unfortunately while the latest compose succeeded, actually trying to
use the installer resulted in a python traceback pretty much immediately

   https://bugzilla.redhat.com/show_bug.cgi?id=1461469

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide: where for art thou? (why no rawhide composes recently)

[Martin Kolman]

On Tue, 2017-06-13 at 22:47 +0100, Peter Robinson wrote:
> > > > > Greetings.
> > > > > 
> > > > > Some folks may have noticed that there have been no completed
> > > > > rawhide
> > > > > composes in a while (13 days as of today).
> > > > > 
> > > > > This has been due to a variety of bugs and issues, along with
> > > > > pungi now
> > > > > failing composes that don't have all required release
> > > > > blocking items.
> > > > 
> > > > Is there a way we can loosen that up for rawhide and have it
> > > > tightened
> > > > down for branched. I think it's worth while to have at least a
> > > > flow of
> > > > the everything repository out on a regular basis like the old
> > > > pre
> > > > pungi-4 use to do.
> > > 
> > > Possibly yes. But I think the idea was to do it for rawhide as
> > > well and
> > > thus always be at least "Alpha" quality so we could not do alpha
> > > anymore.
> > 
> > That was my take on the "no more alphas". If we can't do this then
> > we
> > are going to have to look at doing an alpha in the schedule because
> > it
> > is clear we aren't able to stabilize enough for that promise to
> > exist.
> 
> For actual artifacts such as cloud/disk/installer images I agree but
> at least pushing out individual packages so people can do "dnf
> upgrade" picks up issues such as dependency issues that also kill the
> compose and allows people to still test explicit parts and have the
> composite parts of a "rolling release" and get things fixed I
> feel
> that's better than dragging everything to a blinding halt like we
> have
> for the last 13... or is it 14 days?
> 
> I can't help but feel this is like British politics is ATM where
> people are claiming everything is "strong and stable" while the
> wheels
> have fallen off and are rolling down the road. I don't think pushing
> out the Everything repo stops the "kill off Alpha" process from
> happening, in fact I believe it means it's more likely to happen
> because if the last two weeks shows anything all that happens is that
> if we wait for a "everything is perfect ship it" we never will and
> because nothing is shipped nothing is tested
I agree with that - for example the Anaconda installer CI broke due to
a rawhide package change in the last "working" compose. There is a
fixed package available and built in Koji, but it's not reaching the
Rawhide repos due to the broken composes. In this case we can fix the
CI by using a COPR build of the package, but in other cases it could be
more difficult.

BTW, when was the change to only release packages to repositories once
a compose succeeds introduced ? I kinda always thought packages go to
the Rawhide repos almost immediately - mirroring machinery permitting.


>  so once we get to the
> "computer thinks it's good" we can get to the actual testing and then
> we get to "what the hell changed in the last two weeks broke X, Y and
> Z, are they related or completely independent?" process.
> 
> I think the all or nothing actually makes it less likely for us to
> ever ship anything! I don't think shipping the traditional
> "Everything" repo breaks the "kill Alpha" proposal, I think we need
> to
> be pragmatic and realise that people actually consuming content helps
> that.
> 
> Peter "yes I live in the as strong and stable as a house of cards
> country so I can joke/comment on it" R
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide: where for art thou? (why no rawhide composes recently)

[Matthew Miller]

On Tue, Jun 13, 2017 at 03:31:37PM -0600, Kevin Fenzi wrote:
> > I know we're already at new-deliverable explosion, but this seems like
> > a place where it'd be nice to have Rawhide and Bikeshed (or whatever we
> > want to call "tested and believed-to-be basically functional Rawhide").
> (Side note: I am not sure why we can't just make these the same thing...
> ie, always have rawhide basically functional).

Well... I guess because the reason it isn't functional right now is
non-trivial?

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide: where for art thou? (why no rawhide composes recently)

[Peter Robinson]

 Greetings.

 Some folks may have noticed that there have been no completed rawhide
 composes in a while (13 days as of today).

 This has been due to a variety of bugs and issues, along with pungi now
 failing composes that don't have all required release blocking items.
>>>
>>> Is there a way we can loosen that up for rawhide and have it tightened
>>> down for branched. I think it's worth while to have at least a flow of
>>> the everything repository out on a regular basis like the old pre
>>> pungi-4 use to do.
>>
>> Possibly yes. But I think the idea was to do it for rawhide as well and
>> thus always be at least "Alpha" quality so we could not do alpha anymore.
>
> That was my take on the "no more alphas". If we can't do this then we
> are going to have to look at doing an alpha in the schedule because it
> is clear we aren't able to stabilize enough for that promise to exist.

For actual artifacts such as cloud/disk/installer images I agree but
at least pushing out individual packages so people can do "dnf
upgrade" picks up issues such as dependency issues that also kill the
compose and allows people to still test explicit parts and have the
composite parts of a "rolling release" and get things fixed I feel
that's better than dragging everything to a blinding halt like we have
for the last 13... or is it 14 days?

I can't help but feel this is like British politics is ATM where
people are claiming everything is "strong and stable" while the wheels
have fallen off and are rolling down the road. I don't think pushing
out the Everything repo stops the "kill off Alpha" process from
happening, in fact I believe it means it's more likely to happen
because if the last two weeks shows anything all that happens is that
if we wait for a "everything is perfect ship it" we never will and
because nothing is shipped nothing is tested so once we get to the
"computer thinks it's good" we can get to the actual testing and then
we get to "what the hell changed in the last two weeks broke X, Y and
Z, are they related or completely independent?" process.

I think the all or nothing actually makes it less likely for us to
ever ship anything! I don't think shipping the traditional
"Everything" repo breaks the "kill Alpha" proposal, I think we need to
be pragmatic and realise that people actually consuming content helps
that.

Peter "yes I live in the as strong and stable as a house of cards
country so I can joke/comment on it" R
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide: where for art thou? (why no rawhide composes recently)

[Stephen John Smoogen]

On 13 June 2017 at 17:28, Kevin Fenzi  wrote:
> On 06/13/2017 08:10 AM, Peter Robinson wrote:
>> On Tue, Jun 13, 2017 at 2:11 PM, Kevin Fenzi  wrote:
>>> Greetings.
>>>
>>> Some folks may have noticed that there have been no completed rawhide
>>> composes in a while (13 days as of today).
>>>
>>> This has been due to a variety of bugs and issues, along with pungi now
>>> failing composes that don't have all required release blocking items.
>>
>> Is there a way we can loosen that up for rawhide and have it tightened
>> down for branched. I think it's worth while to have at least a flow of
>> the everything repository out on a regular basis like the old pre
>> pungi-4 use to do.
>
> Possibly yes. But I think the idea was to do it for rawhide as well and
> thus always be at least "Alpha" quality so we could not do alpha anymore.

That was my take on the "no more alphas". If we can't do this then we
are going to have to look at doing an alpha in the schedule because it
is clear we aren't able to stabilize enough for that promise to exist.

> kevin
>
>
>
>
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
>



-- 
Stephen J Smoogen.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide: where for art thou? (why no rawhide composes recently)

[Kevin Fenzi]

On 06/13/2017 08:43 AM, Matthew Miller wrote:
> On Tue, Jun 13, 2017 at 03:10:58PM +0100, Peter Robinson wrote:
>>> This has been due to a variety of bugs and issues, along with pungi now
>>> failing composes that don't have all required release blocking items.
>>
>> Is there a way we can loosen that up for rawhide and have it tightened
>> down for branched. I think it's worth while to have at least a flow of
>> the everything repository out on a regular basis like the old pre
>> pungi-4 use to do.
> 
> I know we're already at new-deliverable explosion, but this seems like
> a place where it'd be nice to have Rawhide and Bikeshed (or whatever we
> want to call "tested and believed-to-be basically functional Rawhide").

(Side note: I am not sure why we can't just make these the same thing...
ie, always have rawhide basically functional).

> That doesn't seem imminent, though, so Peter's suggestion seems
> reasonable. Then maybe we could ask Jan as FPgM to keep track of what's
> working and what's broken and help make sure the right people are on
> it.

Well, it's sometimes not easy to tell where something is broken, it
takes a human to look and dig around and see where the failure is.

Example: Today's rawhide compose I had hopes for failed.

Turns out it was a koji bug we already fixed and we didn't hit it
yesterday because we didn't have the same fixed koji version on all the
builders. It took me and Patrick a while to track down what was
happening there. There's also another compose running now. Fingers
crossed. :)

kevin






signature.asc
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide: where for art thou? (why no rawhide composes recently)

[Kevin Fenzi]

On 06/13/2017 08:10 AM, Peter Robinson wrote:
> On Tue, Jun 13, 2017 at 2:11 PM, Kevin Fenzi  wrote:
>> Greetings.
>>
>> Some folks may have noticed that there have been no completed rawhide
>> composes in a while (13 days as of today).
>>
>> This has been due to a variety of bugs and issues, along with pungi now
>> failing composes that don't have all required release blocking items.
> 
> Is there a way we can loosen that up for rawhide and have it tightened
> down for branched. I think it's worth while to have at least a flow of
> the everything repository out on a regular basis like the old pre
> pungi-4 use to do.

Possibly yes. But I think the idea was to do it for rawhide as well and
thus always be at least "Alpha" quality so we could not do alpha anymore.

kevin





signature.asc
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide: where for art thou? (why no rawhide composes recently)

[Kaleb S. KEITHLEY]

You keep using that word — where for [sic] — I do not think it means 
what you think it means. (As inconceivable as it may seem.)


--

Kaleb
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide: where for art thou? (why no rawhide composes recently)

[Colin Walters]

On Tue, Jun 13, 2017, at 10:43 AM, Matthew Miller wrote:

> I know we're already at new-deliverable explosion, but this seems like
> a place where it'd be nice to have Rawhide and Bikeshed (or whatever we
> want to call "tested and believed-to-be basically functional Rawhide").
> That doesn't seem imminent, though, so Peter's suggestion seems
> reasonable. Then maybe we could ask Jan as FPgM to keep track of what's
> working and what's broken and help make sure the right people are on
> it.

An alternative to creating *even more* streams is to *revert* broken
things in existing streams.  lorax didn't work on arm?   Revert, don't
just file a bugzilla and have everyone sitting around wait for a patch.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide: where for art thou? (why no rawhide composes recently)

[Matthew Miller]

On Tue, Jun 13, 2017 at 03:10:58PM +0100, Peter Robinson wrote:
> > This has been due to a variety of bugs and issues, along with pungi now
> > failing composes that don't have all required release blocking items.
> 
> Is there a way we can loosen that up for rawhide and have it tightened
> down for branched. I think it's worth while to have at least a flow of
> the everything repository out on a regular basis like the old pre
> pungi-4 use to do.

I know we're already at new-deliverable explosion, but this seems like
a place where it'd be nice to have Rawhide and Bikeshed (or whatever we
want to call "tested and believed-to-be basically functional Rawhide").
That doesn't seem imminent, though, so Peter's suggestion seems
reasonable. Then maybe we could ask Jan as FPgM to keep track of what's
working and what's broken and help make sure the right people are on
it.


-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Rawhide: where for art thou? (why no rawhide composes recently)

[Peter Robinson]

On Tue, Jun 13, 2017 at 2:11 PM, Kevin Fenzi  wrote:
> Greetings.
>
> Some folks may have noticed that there have been no completed rawhide
> composes in a while (13 days as of today).
>
> This has been due to a variety of bugs and issues, along with pungi now
> failing composes that don't have all required release blocking items.

Is there a way we can loosen that up for rawhide and have it tightened
down for branched. I think it's worth while to have at least a flow of
the everything repository out on a regular basis like the old pre
pungi-4 use to do.

> Here's a partial list:
>
> 2017-06-01 - lorax traceback, bug 1457055
> 2017-06-02 - another lorax issue, bug 1457906
> 2017-06-03 - cloud base failed in anaconda, bug 1458509
> 2017-06-04 - ditto
> 2017-06-05 - ditto
> 2017-06-06 - pungi bug - https://pagure.io/pungi/issue/641
> 2017-06-07 - ditto
> 2017-06-08 - ditto
> 2017-06-09 - ditto
> 2017-06-10 - ditto
> 2017-06-11 - ditto
> 2017-06-12 - ditto
> 2017-06-12.1 - pungi bug fixed, but hit libgtop2 broken deps in metacity
> that failed the comppose. I fixed those (and control-center) last night.
> 2017-06-13 - still running, cross your fingers.
>
> All of this has been made a bit worse by us having some storage slowness
> which means you can really only do about 2 rawhide compose attempts a
> day. Thats being worked on and hopefully we will get it fixed soon.
>
> You can also see here there were some gaps where we didn't yet have the
> bug tracked down or didn't yet have a fix in place. We need to try and
> do better there. (I was out on vacation last week, so it can't always be
> me).
>
> Anyhow, hopefully we will have a rawhide compose today, and if not I
> will keep poking it to get it going...
>
> kevin
>
>
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
>
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org