Re: git-ubuntu 0.7.3 in edge/candidate/stable channels [Was Re: git-ubuntu 0.7.1 in edge/candidate/stable channels]

[Nish Aravamudan]

On Tue, Mar 27, 2018 at 3:10 PM, Nish Aravamudan
 wrote:
> On Fri, Mar 16, 2018 at 11:39 AM, Nish Aravamudan
>  wrote:
>> On Thu, Mar 15, 2018 at 2:45 PM, Nish Aravamudan
>>  wrote:
>>> On 12.03.2018 [11:59:11 -0700], Nish Aravamudan wrote:
 On Fri, Mar 2, 2018 at 2:00 PM, Nish Aravamudan
  wrote:
>>
>> 
>>
 Once that is done, we will start the 'keep-up' script again, which
 will spend some time catching up the repositories based upon Launchpad
 publishes since the mass reimport.
>>>
>>> This has been started and is churning through roughly 11 days of
>>> publishing backlog.
>>>
 Finally, we will then look to bump our phasing to 2% (of main), to
 approximate the disk usage as the phasing increases.
>>>
>>> I am looking into this step now.
>>
>> The keep-up script ... well, caught up, and we successfully increased
>> our phasing of main to 2%. We are now increasing again to 10%.
>
> We successfully went through 10% and 20% of main, and have updated the
> default Git repository alias for imported packages to point to the
> imported repositories (so if you clone them, we should see minimal
> on-disk usage in Launchpad).
>
> We are now starting a 50% of main run, which I expect to take a while.

I'm excited to announce that we have completed our first 100% phasing
of main round of imports.

We have a handful of failures that are being monitored in
https://bugs.launchpad.net/usd-importer/+bug/1754898, but for those
packages not in our blacklist, we expect all of them to be imported
and kept current now. They are also setup to be the default Git
repository for the corresponding source package.

Thanks!
-Nish

>
> Thanks,
> Nish

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: git-ubuntu 0.7.3 in edge/candidate/stable channels [Was Re: git-ubuntu 0.7.1 in edge/candidate/stable channels]

[Nish Aravamudan]

On Fri, Mar 16, 2018 at 11:39 AM, Nish Aravamudan
 wrote:
> On Thu, Mar 15, 2018 at 2:45 PM, Nish Aravamudan
>  wrote:
>> On 12.03.2018 [11:59:11 -0700], Nish Aravamudan wrote:
>>> On Fri, Mar 2, 2018 at 2:00 PM, Nish Aravamudan
>>>  wrote:
>
> 
>
>>> Once that is done, we will start the 'keep-up' script again, which
>>> will spend some time catching up the repositories based upon Launchpad
>>> publishes since the mass reimport.
>>
>> This has been started and is churning through roughly 11 days of
>> publishing backlog.
>>
>>> Finally, we will then look to bump our phasing to 2% (of main), to
>>> approximate the disk usage as the phasing increases.
>>
>> I am looking into this step now.
>
> The keep-up script ... well, caught up, and we successfully increased
> our phasing of main to 2%. We are now increasing again to 10%.

We successfully went through 10% and 20% of main, and have updated the
default Git repository alias for imported packages to point to the
imported repositories (so if you clone them, we should see minimal
on-disk usage in Launchpad).

We are now starting a 50% of main run, which I expect to take a while.

Thanks,
Nish

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: git-ubuntu 0.7.3 in edge/candidate/stable channels [Was Re: git-ubuntu 0.7.1 in edge/candidate/stable channels]

[Nish Aravamudan]

On Thu, Mar 15, 2018 at 2:45 PM, Nish Aravamudan
 wrote:
> On 12.03.2018 [11:59:11 -0700], Nish Aravamudan wrote:
>> On Fri, Mar 2, 2018 at 2:00 PM, Nish Aravamudan
>>  wrote:



>> Once that is done, we will start the 'keep-up' script again, which
>> will spend some time catching up the repositories based upon Launchpad
>> publishes since the mass reimport.
>
> This has been started and is churning through roughly 11 days of
> publishing backlog.
>
>> Finally, we will then look to bump our phasing to 2% (of main), to
>> approximate the disk usage as the phasing increases.
>
> I am looking into this step now.

The keep-up script ... well, caught up, and we successfully increased
our phasing of main to 2%. We are now increasing again to 10%.

Thanks,
Nish

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: git-ubuntu 0.7.3 in edge/candidate/stable channels [Was Re: git-ubuntu 0.7.1 in edge/candidate/stable channels]

[Nish Aravamudan]

On 12.03.2018 [11:59:11 -0700], Nish Aravamudan wrote:
> On Fri, Mar 2, 2018 at 2:00 PM, Nish Aravamudan
>  wrote:
> > On Thu, Mar 1, 2018 at 5:20 PM, Nish Aravamudan
> >  wrote:
> >> On 28.02.2018 [21:33:38 -0800], Nish Aravamudan wrote:
> >>> Hello all!
> >>>
> >>> I am  happy to announce the release of git-ubuntu 0.7.1 to all channels
> >>> today.
> >>
> >> I encountered some unfortunate issues with snapcraft that hit
> >> xenial-updates just as I was building (LP: #1752481) and then realized
> >> we needed to snap the archive keyrings in order to verify recent archive
> >> files (LP: #1752656). I then made a typo fixing the latter, so we're now
> >> at 0.7.3. This should build successfully in all channels shortly.
> >>
> >> Thanks to Sergio Schvezov for the snapcraft assist and Steve Langasek
> >> for the keyring suggestion.
> >
> > Well, it didn't quite go as planned. I found another bug (so we're at
> > 0.7.4) and also the snapcraft fix didn't resolve all the issues. So
> > I'm temporarily using a PPA that reverts the xenial-updates of
> > snapcraft back to 2.35.
> >
> > Kyle Fazzari is working on a proper fix, but if it's not able to get
> > done soon, we might back out the snapcraft SRU in xenial-updates.
> 
> For completeness, we ended up backing out the snapcraft SRU via a
> re-upload of 2.35 to xenial-updates. The snapcraft team is working on
> fixes to the bugs found.
> 
> Meanwhile, a mass reimport ran last week and the existing whitelist +
> 1% of main was reimported. This resulted in 905 successful imports and
> 5 failed imports. The latter 5 will presumably be broken for a while,
> as we focus on upping our phasing before resolving those issues (LP:
> #1754898).

I have removed the repositories for the failure cases. They would not
update with the normal 'keep the repositories current' anyways, and we
would rather have consistent repository contents.

> I am working right now on an issue with our default repository
> repointing script.

This was PEBKAC.

> Additionally, roughly 150 manual imports remain in the
> ~usd-import-team space, but those source packages were never added to
> the whitelist. They need to be reimported, as well (and added to the
> whitelist). I am waiting to ensure that there are no pending MPs for
> those source packages before reimporting.

This was completed this morning.

> Once that is done, we will start the 'keep-up' script again, which
> will spend some time catching up the repositories based upon Launchpad
> publishes since the mass reimport.

This has been started and is churning through roughly 11 days of
publishing backlog.

> Finally, we will then look to bump our phasing to 2% (of main), to
> approximate the disk usage as the phasing increases.

I am looking into this step now.

Thanks,
Nish

-- 
Nishanth Aravamudan
Ubuntu Server
Canonical Ltd

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: git-ubuntu 0.7.3 in edge/candidate/stable channels [Was Re: git-ubuntu 0.7.1 in edge/candidate/stable channels]

[Nish Aravamudan]

On Fri, Mar 2, 2018 at 2:00 PM, Nish Aravamudan
 wrote:
> On Thu, Mar 1, 2018 at 5:20 PM, Nish Aravamudan
>  wrote:
>> On 28.02.2018 [21:33:38 -0800], Nish Aravamudan wrote:
>>> Hello all!
>>>
>>> I am  happy to announce the release of git-ubuntu 0.7.1 to all channels
>>> today.
>>
>> I encountered some unfortunate issues with snapcraft that hit
>> xenial-updates just as I was building (LP: #1752481) and then realized
>> we needed to snap the archive keyrings in order to verify recent archive
>> files (LP: #1752656). I then made a typo fixing the latter, so we're now
>> at 0.7.3. This should build successfully in all channels shortly.
>>
>> Thanks to Sergio Schvezov for the snapcraft assist and Steve Langasek
>> for the keyring suggestion.
>
> Well, it didn't quite go as planned. I found another bug (so we're at
> 0.7.4) and also the snapcraft fix didn't resolve all the issues. So
> I'm temporarily using a PPA that reverts the xenial-updates of
> snapcraft back to 2.35.
>
> Kyle Fazzari is working on a proper fix, but if it's not able to get
> done soon, we might back out the snapcraft SRU in xenial-updates.

For completeness, we ended up backing out the snapcraft SRU via a
re-upload of 2.35 to xenial-updates. The snapcraft team is working on
fixes to the bugs found.

Meanwhile, a mass reimport ran last week and the existing whitelist +
1% of main was reimported. This resulted in 905 successful imports and
5 failed imports. The latter 5 will presumably be broken for a while,
as we focus on upping our phasing before resolving those issues (LP:
#1754898).

I am working right now on an issue with our default repository
repointing script.

Additionally, roughly 150 manual imports remain in the
~usd-import-team space, but those source packages were never added to
the whitelist. They need to be reimported, as well (and added to the
whitelist). I am waiting to ensure that there are no pending MPs for
those source packages before reimporting.

Once that is done, we will start the 'keep-up' script again, which
will spend some time catching up the repositories based upon Launchpad
publishes since the mass reimport.

Finally, we will then look to bump our phasing to 2% (of main), to
approximate the disk usage as the phasing increases.

Thanks for your patience as we continue to work through this process.

-Nish

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel